Integrating A Maven-Backend- With A Nodjs/Grunt-Fronted-Project

Frontend-Development With Nodjs and Grunt

As I already wrote in a previous article, frontend-development is mostly done with Nodjs and Grunt nowadays.
As I am planing to base the frontend of my next Spring-Application on Bootstrap, I was looking for a way to integrate my backend, which is build using Spring and Thymeleaf and managed with Maven, with a frontend, which is based on Bootstrap and, hence, build with Nodjs and Grunt.

Integrate The Frontend-Build Into The Maven-Build-Process

As I found out, one can integrate a npm-based build into a maven project with the help of the frontend-maven-plugin.
This plugin automates the managment of Nodjs and its libraries and ensures that the version of Node and NPM being run is the same in every build environment.
As a backend-developer, you do not have to install any of the frontend-tools manualy.
Because of that, this plugin is ideal to integrate a separately developed frontend into a maven-build, without bothering the backend-developers with details of the frontend-build-process.

Seperate The Frontend-Project From The Maven-Based Backend-Project

The drawback with this approach is, that the backend- and the frontend-project are tightly coupled.
You can configure the frontend-maven-plugin to use a separate subdirectory as working-directory (for example src/main/frontend) and utilize this to separate the frontend-project in its own repository (for example by using the submodule-functions of git).
But the grunt-tasks, that you call in the frontend-project through the frontend-maven-plugin, must be defined in that project.

Since I am planing to integrate a ‐ slightly modified ‐ version of Bootstrap as frontend into my project, that would mean that I have to mess around with the configuration of the Bootstrap-project a lot.
But that is not a very good idea, because it hinders upgrades of the Bootstrap-base, because merge-conflicts became more and more likely.

So, I decided to program a special Gruntfile.js, that resides in the base-folder of my Maven-project and lets me redefine and call tasks of a separated frontend-project in a subdirectory.

Redefine And Call Tasks Of An Included Gruntfile From A Sub-Project

As it turned out, there are several npm-plugins for managing and building sub-projects (like grunt-subgrunt or grunt-recurse) or including existing Gruntfiles from sub-projects (like grunt-load-gruntfile), but none of them lets you redefine tasks of the subproject before calling them.

I programmed a simple Gruntfile, that lets you do exactly this:


module.exports = function(grunt) {

  grunt.loadNpmTasks('grunt-newer');

  grunt.registerTask('frontend','Build HTML & CSS for Frontend', function() {
    var
    done = this.async(),
    path = './src/main/frontend';

    grunt.util.spawn({
      cmd: 'npm',
      args: ['install'],
      opts: { cwd: path, stdio: 'inherit' }
    }, function (err, result, code) {
      if (err || code > 0) {
        grunt.fail.warn('Failed installing node modules in "' + path + '".');
      }
      else {
        grunt.log.ok('Installed node modules in "' + path + '".');
      }

      process.chdir(path);
      require(path + '/Gruntfile.js')(grunt);
      grunt.task.run('newer:copy');
      grunt.task.run('newer:less');
      grunt.task.run('newer:svgstore');

      done();
    });
  });


  grunt.registerTask('default', [ 'frontend' ]);

};

This Gruntfile loads the npm-taks grunt-newer.
Then, it registers a grunt-task called frontend, that loads the dependencies of the specified sub-project, read in its Gruntfile and runs redefined versions of the tasks copy, less and svgstore, which are defined in the sub-project.
The sub-project itself does not register grunt-newer itself.
This is done in this parent-project, to demonstrate how to register additional grunt-plugins and redefine tasks of the sub-project without touching it at all.

The separated frontend-project can be used by the frontend-team to develop the temlates, needed by the backend-developers, without any knowledge of the maven-project.
The frontend-project is then included into the backend, which is managed by maven, and can be used by the backend-developers without the need to know anything about the techniques that were used to develop the templates.

The whole example can be browsed at juplo.de/gitweb or cloned with:


git clone http://juplo.de/git/examples/maven-grunt-integration

Be sure to checkout the tag 2.0.0 for the corresponding version after the cloning, in case i add more commits to demonstrate other stuff.
Also, you have to init and clone the submodule after checkout:


git submodule init
git submodule update

If you run mvn jetty:run, you will notice, that the frontend-maven-plugin will automatically download Nodejs into a the folder node of the parent-project.
Afterwards, the dependencies of the parent-project are downloaded in the folder node_modules of the parent-project and the dpendencies of the sub-project are downloaded in the folder src/main/frontend/node_modules and the sub-project is build automatically in the folder src/main/frontend/dist, which is included into the directory-tree that is served by the jetty-maven-plugin.

The sub-project is fully usable standalone to drive the development of the frontend separately.
You can read more about it in this previous article.

Conclusion

In this article, I showed how to integrate a separately developed frontend-project into a backend-project managed by Maven.
This enables you to separate the development of the layout and the logic of a classic ROCA-project nearly totally.

Serving Static HTML With Nodjs And Grunt For Template-Development

A Simple Nodejs/Grunt-Development-Environment for static HTML-Templates

Nowadays, frontend-development is mostly done with Nodjs and Grunt.
On npm, there are plenty of useful plugin’s, that ease the development of HTML and CSS.
For example grunt-contrib-less to automate the compilation of LESS-sourcecode to CSS, or grunt-svgstore to pack several SVG-graphics in a single SVG-sprite.

Because of that, I decided to switch to Nodejs and Grunt to develop the HTML- and CSS-Markup for the templates, that I need for my Spring/Thymeleaf-Applications.
But as with everything new, I had some hard work, to plug together what I needed.
In this article I want to share, how I have set up a really minimalistic, but powerful development-environment for static HTML-templates, that suites all of my initial needs.

This might not be the best solutions, but it is a good starting point for beginners like me and it is here to be improved through your feedback!

You can browse the example-development-environment on juplo.de/gitweb, or clone it with:


git clone http://juplo.de/git/examples/template-development

After installing npm you have to fetch the dependencies with:


npm install

Than you can fire up a build with:


grunt

…or start a webserver for development with:


git run-server

Serving The HTML and CSS For Local Development

The hardest part while putting together the development-environment was my need to automatically build the static HTML and CSS after file-changes and serve them via a local webserver.
As I wrote in an earlier article, I often stumble over problems, that arise from the Same-origin policy when accessing the files locally through file:///-URI’s).

I was a bit surprised, that I could not find a simple explanation, how to set up a grunt-task to build the project automatically on file-changes and serve the generated HTML and CSS locally.
That is the main reason, why I am writing this explanation now, in order to fill that gap ;)

I realised that goal by implemnting a grunt-task, that spawn’s a process that uses the http-server to serve up the files and combine that task with a common watch-task:


grunt.registerTask('http-server', function() {

  grunt.util.spawn({
    cmd: 'node_modules/http-server/bin/http-server',
    args: [ 'dist' ],
    opts: { stdio: 'inherit' }
  });

});

grunt.registerTask('run-server', [ 'default', 'http-server', 'watch' ]);

The rest of the configuration is really pretty self-explaining.
I just put together the pieces I needed for my template development (copy some static HTML and generate CSS from the LESS-sources) and configured grunt-contrib-watch to rebuild the project automatically, if anything changes.

The result is put under dist/ and is ready to be included in my Spring/Thymeleaf-Application as it is.

Bypassing the Same-Origin-Policy For Local Files During Development

downloadable font: download failed …: status=2147500037

Are you ever stumbled accross weired errors with font-files, that could not be loaded, or SVG-graphics, that are not shown during local development on your machine using file:///-URI’s, though everything works as expected, if you push the content to a webserver and access it via HTTP?
Furthermore, the browsers behave very differently here.
Firefox, for example, just states, that the download of the font failed:


downloadable font: download failed (font-family: "XYZ" style:normal weight:normal stretch:normal src index:0): status=2147500037 source: file:///home/you/path/to/font/xyz.woff

Meanwhile, Chrome just happily uses the same font.
Considering the SVG-graphics, that are not shown, Firefox just does not show them, like it would not be able to at all.
Chrome logs an error:


Unsafe attempt to load URL file:///home/you/path/to/project/img/sprite.svg#logo from frame with URL file:///home/you/path/to/project/templates/layout.html. Domains, protocols and ports must match

…though, no protocol, domain or port is involved.

The Same-Origin Policy

The reason for this strange behavior is the Same-origin policy.
Chrome gives you a hint in this direction with the remark that something does not match.
I found the trail, that lead me to this explanation, while googling for the strange error message, that Firefox gives for the fonts, that can not be loaded.


The Same-origin policy forbids, that locally stored files can access any data, that is stored in a parent-directory.
They only have access to files, that reside in the same directory or in a directory beneath it.

You can read more about that rule on MDN.

I often violate that rule, when developing templates for dynamically rendered pages with Thymeleaf, or similar techniques.
That is, because I like to place the template-files on a subdirectory of the directory, that contains my webapp (src/main/webapp with Maven):


+ src/main/webapp/
  + css/
  + img/
  + fonts/
  + thymeleaf/templates/

I packed a simple example-project for developing static templates with LESS, nodejs and grunt, that shows the problem and the quick solution for Firefox presented later.
You can browse it on my juplo.de/gitweb, or clone it with:


git clone http://juplo.de/git/examples/template-development

Cross-Browser Solution

Unfortunately, there is no simple cross-browser solution, if you want to access your files through file:///-URI’s during development.
The only real solution is, to access your files through the HTTP-protocol, like in production.
If you do not want to do that, the only two cross-browser solutions are, to

  1. turn of the Same-origin policy for local files in all browsers, or
  2. rearrange your files in such a way, that they do not violate the Same-origin policy (as a rule, all resources linked in a HTML-file must reside in the same directory as the file, or beneath it).

The only real cross-browser solution is to circumvent the problem altogether and serve the content with a local webserver, so that you can access it through HTTP, like in production.
You can read how to extend the example-project mentioned above to achieve that goal in a follow up article.

Turn Of Security

Turning of the Same-origin policy is not recommended.
I would only do that, if you only use your browser, to access the HTML-files under development ‐ which I doubt, that it is the case.
Anyway, this is a good quick test to validate, that the Same-origin policy is the source of your problems ‐ if you quickly re-enable it after the validation.

Firefox:
Set security.fileuri.strict_origin_policy to false on the about:config-page.
Chrome:
Restart Chrome with --disable-web-security or --allow-file-access-from-files (for more, see this question on Stackoverflow).

Quick Fix For Firefox

If you develop with Firefox, there is a quick fix, to bypass the Same-origin policy for local files.

As the explanation on MDM stats, a file loaded in a frame shares the same origin as the file, that contains the frameset.
This can be used to bypass the policy, if you place a file with a frameset in the topmost directory of your development-folder and load the template under development through that file.

In my case, the frameset-file looks like this:


<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN" "http://www.w3.org/TR/html4/frameset.dtd">
<html>
  <head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8">
    <title>Frameset to Bypass Same-Origin-Policy
  </head>
  <frameset>
    <frame src="thymeleaf/templates/layout.html">
  </frameset>
</html>

Combining jetty-maven-plugin and wro4j-maven-plugin for Dynamic Reloading of LESS-Resources

Ever searched for a simple configuration, that lets you use your jetty-maven-plugin as you are used to, while working with LESS to simplify your stylesheets?

You cannot do both, use the Client-side mode of LESS to ease development and use the lesscss-maven-plugin to automatically compile the LESS-sources into CSS for production. That does not work, because your stylesheets must be linked in different ways if you are switching between the client-side mode – which is best for development – and the pre-compiled mode – which is best for production. For the client-side mode you need something like:


<link rel="stylesheet/less" type="text/css" href="styles.less" />
<script src="less.js" type="text/javascript"></script>

While, for the pre-compiled mode, you want to link to your stylesheets as usual, with:


<link rel="stylesheet" type="text/css" href="styles.css" />

While looking for a solution to this dilemma, I stumbled accross wro4j. Originally intended, to speed up page-delivery by combining and minimizing multiple resources into one through the use of a servlet-filter, this tool also comes with a maven-plugin, that let you do the same offline, while compiling your webapp.

The idea is, to use the wro4j-maven-plugin to compile and combine your LESS-sources into CSS for production and to use the wro4j filter, to dynamically deliver the compiled CSS while developing. This way, you do not have to alter your HTML-code, when switching between development and production, because you always link to the CSS-files.

So, lets get dirty!

Step 1: Configure wro4j

First, we configure wro4j, like as we want to use it to speed up our page. The details are explained and linked on wro4j’s Getting-Started-Page. In short, we just need two files: wro.xml and wro.properties.

wro.xml

wro.xml tells wro4j, which resources should be combined and how the result should be named. I am using the following configuration to generate all LESS-Sources beneath base/ into one CSS-file called base.css:


<groups xmlns="http://www.isdc.ro/wro">
  <group name="base">
    <css>/less/base/*.less</css>
  </group>

wro4j looks for /less/base/*.less inside the root of the web-context, which is equal to src/main/webapp in a normal maven-project. There are other ways to specifie the resources, which enable you to store them elswhere. But this approach works best for our goal, because the path is understandable for both: the wro4j servlet-filter, we are configuring now for our development-environment, and the wro4j-maven-plugin, that we will configure later for build-time compilation.

wro.properties

wro.properties in short tells wro4j, how or if it should convert the combined sources and how it should behave. I am using the following configuration to tell wro4j, that it should convert *.less-sources into CSS and do that on every request:


managerFactoryClassName=ro.isdc.wro.manager.factory.ConfigurableWroManagerFactory
preProcessors=cssUrlRewriting,lessCssImport
postProcessors=less4j
disableCache=true

First of all we specify the ConfigurableWroManagerFactory, because otherwise, wro4j would not pick up our pre- and post-processor-configuration. This is a little bit confusing, because wro4j is already reading the wro.properties-file – otherwise wro4j would never detect the managerFactoryClassName-directive – and you might think: “Why? He is already interpreting our configuration!” But belive me, he is not! You can read more about that in wro4j’s documentation. The disableCache=true is also crucial, because otherwise, we would not see the changes take effect when developing with jetty-maven-plugin later on. The pre-processors lessCssImport and cssUrlRewriting merge together all our LESS-resources under /less/base/*.less and do some URL-rewriting, in case you have specified paths to images, fonts or other resources inside your LESS-code, to reflect that the resulting CSS is found under /css/base.css and not /css/base/YOURFILE.css like the LESS-resources.

You can do much more with your resources here, for example minimizing. Also, there are countless configuration options to fine-tune the behaviour of wro4j. But for our goal, we are now only intrested in the compilation of our LESS-sources.

Step 2: Configure the wro4j servlet-filter

Configuring the filter in the web.xml is easy. It is explained in wro4j’s installation-insctuctions. But the trick is, that we do not want to configure that filter for the production-version of our webapp, because we want to compile the resources offline, when the webapp is build. To acchieve this, we can use the <overrideDescriptor>-Parameter of the jetty-maven-plugin.

<overrideDescriptor>

This parameter lets you specify additional configuration options for the web.xml of your webapp. I am using the following configuration for my jetty-maven-plugin:


<plugin>
  <groupId>org.eclipse.jetty</groupId>
  <artifactId>jetty-maven-plugin</artifactId>
  <configuration>
    <webApp>
      <overrideDescriptor>${project.basedir}/src/test/resources/jetty-web.xml</overrideDescriptor>
    </webApp>
  </configuration>
  <dependencies>
    <dependency>
      <groupId>ro.isdc.wro4j</groupId>
      <artifactId>wro4j-core</artifactId>
      <version>${wro4j.version}</version>
    </dependency>
    <dependency>
      <groupId>ro.isdc.wro4j</groupId>
      <artifactId>wro4j-extensions</artifactId>
      <version>${wro4j.version}</version>
      <exclusions>
        <exclusion>
          <groupId>javax.servlet</groupId>
          <artifactId>servlet-api</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.apache.commons</groupId>
          <artifactId>commons-lang3</artifactId>
        </exclusion>
        <exclusion>
          <groupId>commons-io</groupId>
          <artifactId>commons-io</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.springframework</groupId>
          <artifactId>spring-web</artifactId>
        </exclusion>
        <exclusion>
          <groupId>com.google.code.gson</groupId>
          <artifactId>gson</artifactId>
        </exclusion>
        <exclusion>
          <groupId>com.google.javascript</groupId>
          <artifactId>closure-compiler</artifactId>
        </exclusion>
        <exclusion>
          <groupId>com.github.lltyk</groupId>
          <artifactId>dojo-shrinksafe</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.jruby</groupId>
          <artifactId>jruby-core</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.jruby</groupId>
          <artifactId>jruby-stdlib</artifactId>
        </exclusion>
        <exclusion>
          <groupId>me.n4u.sass</groupId>
          <artifactId>sass-gems</artifactId>
        </exclusion>
        <exclusion>
          <groupId>nz.co.edmi</groupId>
          <artifactId>bourbon-gem-jar</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.codehaus.gmaven.runtime</groupId>
          <artifactId>gmaven-runtime-1.7</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.webjars</groupId>
          <artifactId>jshint</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.webjars</groupId>
          <artifactId>emberjs</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.webjars</groupId>
          <artifactId>handlebars</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.webjars</groupId>
          <artifactId>coffee-script</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.webjars</groupId>
          <artifactId>jslint</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.webjars</groupId>
          <artifactId>json2</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.webjars</groupId>
          <artifactId>jquery</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
  </dependencies>
</plugin>

The dependencies to wro4j-core and wro4j-extensions are needed by jetty, to be able to enable the filter defined below. Unfortunatly, one of the transitive dependencies of wro4j-extensions triggers an uggly error when running the jetty-maven-plugin. Therefore, all unneeded dependencies of wro4j-extensions are excluded, as a workaround for this error/bug.

jetty-web.xml

And my jetty-web.xml looks like this:


<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
  version="2.5">
  <filter>
    <filter-name>wro</filter-name>
    <filter-class>ro.isdc.wro.http.WroFilter</filter-class>
  </filter>
  <filter-mapping>
    <filter-name>wro</filter-name>
    <url-pattern>*.css</url-pattern>
  </filter-mapping>
</web-app>

The filter processes any URI’s that end with .css. This way, the wro4j servlet-filter makes base.css available under any path, because for exampl /base.css, /css/base.css and /foo/bar/base.css all end with .css.

This is all, that is needed to develop with dynamically reloadable compiled LESS-resources. Just fire up your browser and browse to /what/you/like/base.css. (But do not forget to put some LESS-files in src/main/webapp/less/base/ first!)

Step 3: Install wro4j-maven-plugin

All that is left over to configure now, is the build-process. If you would build and deploy your webapp now, the CSS-file base.css would not be generated and the link to your stylesheet, that already works in our jetty-maven-plugin environment would point to a 404. Hence, we need to set up the wro4j-maven-plugin. I am using this configuration:


<plugin>
  <groupId>ro.isdc.wro4j</groupId>
  <artifactId>wro4j-maven-plugin</artifactId>
  <version>${wro4j.version}</version>
  <configuration>
    <wroManagerFactory>ro.isdc.wro.maven.plugin.manager.factory.ConfigurableWroManagerFactory</wroManagerFactory>
    <cssDestinationFolder>${project.build.directory}/${project.build.finalName}/css/</cssDestinationFolder>
  </configuration>
  <executions>
    <execution>
      <phase>prepare-package</phase>
      <goals>
        <goal>run</goal>
      </goals>
    </execution>
  </executions>
</plugin>

I connected the run-goal with the package-phase, because the statically compiled CSS-file is needed only in the final war. The ConfigurableWroManagerFactory tells wro4j, that it should look up further configuration options in our wro.properties-file, where we tell wro4j, that it should compile our LESS-resources. The <cssDestinationFolder>-tag tells wro4j, where it should put the generated CSS-file. You can adjust that to suite your needs.

That’s it: now the same CSS-file, which is created on the fly by the wro4j servlet-filter when using mvn jetty:run and, thus, enables dynamic reloading of our LESS-resources, is generated during the build-process by the wro4j-maven-plugin.

Cleanup and further considerations

lesscss-maven-plugin

If you already compile your LESS-resources with the lesscss-maven-plugin, you can stick with it and skip step 3. But I strongly recommend giving wro4j-maven-plugin a try, because it is a much more powerfull tool, that can speed up your final webapp even more.

Clean up your mess

With a configuration like the above one, your LESS-resources and wro4j-configuration-files will be packed into your production-war. That might be confusing later, because neither wro4j nor LESS is used in the final war. You can add the following to your pom.xml to exclude these files from your war for the sake of clarity:


<plugin>
  <artifactId>maven-war-plugin</artifactId>
  <configuration>
    <warSourceExcludes>
      WEB-INF/wro.*,
      less/**
    </warSourceExcludes>
  </configuration>
</plugin>

What’s next?

We only scrached the surface, of what can be done with wro4j. Based on this configuration, you can easily enable additional features to fine-tune your final build for maximum speed. You really should take a look at the list of available Processors!