DigitalJoel

2014/01/27

Adding Security to a Partially Exposed Web Service

Filed under: development, java, spring, spring framwork, spring-mvc — Tags: , — digitaljoel @ 10:42 pm

In my previous post I talked about adding some conditional security to a web service by only exposing certain methods and model representations using the new Conditional annotation and a HandlerInterceptor in a Spring 4 based Spring Boot app. Tonight I decided to add some real Spring Security magic to it.

First, add the Spring Security dependency. I took this right from the Spring Security guide on the Spring.io website.

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
            <version>0.5.0.M6</version>
        </dependency>

Then I added the following new Configuration class to my existing Application.

  @Configuration
  @EnableWebSecurity
  static class WebSecurityConfig extends WebSecurityConfigurerAdapter {
    @Override
    protected void configure(AuthenticationManagerBuilder auth) throws Exception {
        auth.inMemoryAuthentication().withUser("user").password("password").roles("USER");
        auth.inMemoryAuthentication().withUser("admin").password("password").roles("SUPER");
    }
  }

Next, I modified my HandlerInterceptor and it ended up as follows:

public class PublicHandlerInterceptor extends HandlerInterceptorAdapter {

  @Override
  public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
    Object principal = SecurityContextHolder.getContext().getAuthentication().getPrincipal();
    if ( handler instanceof HandlerMethod ) {
      HandlerMethod method = (HandlerMethod)handler;
      if ( method.getMethodAnnotation(Public.class) != null
          && hasAnyRole( (User)principal, method.getMethodAnnotation(Public.class).forRoles())) {
        return true;
      }
      response.setStatus(404);
    }
    return false;
  }
  
  private boolean hasAnyRole( User principal, String[] rolesStrings ) {
    if ( rolesStrings == null || rolesStrings.length == 0 ) {
      return true;
    }
    Set<String> roles = Sets.newHashSet(rolesStrings);
    for ( GrantedAuthority auth : principal.getAuthorities() ) {
      if ( roles.contains(auth.getAuthority())) {
        return true;
      }
    }
    return false;
  }
}

In the previous iteration I was simply looking for the Public annotation. Now I am looking for a parameter on that annotation. The parameter defaults to empty, which behaves just like the previous iteration of the project, but now you can specify that it should only be public for certain roles. Obviously, this necessitated a change to the Public annotation, as follows:

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface Public {
  String[] forRoles() default {};
}

Finally, I modified the usage of the annotation to test out the new functionality:

  @Public(forRoles="ROLE_USER")

You should also be able to pass an array of role names to the forRoles parameter of the annotation.

The Spring Security filters are executed before my HandlerInterceptor so the user would have gone through all the authentication checks by that point. Then, in my HandlerInterceptor, rather than let the user know that there is an endpoint at the given location they just have to try harder to hack it, they get a 404 if they are not allowed to access it.

I suspect you could probably get the same by using a custom handler for an AuthorizationException (or whatever spring-security throws) and using a spring security method annotation to check the role and return a 404 from the handler instead of the standard response but I wanted to build on the previous example and keep the conditional behavior.

Advertisements

2014/01/10

Exposing a read-only view of a Spring-MVC web service

Alright, so this is actually more flexible than just a read-only view, but that was the case that prompted me to play around with things so that’s where I’m starting. I was partially inspired by a co-worker’s blog entry regarding creating resource filters with jersey and jaxrs 2.0.

So down to the scenario. I have a simple CRUD webservice that I’ve implemented in Spring-MVC. For my demonstration I used Spring Boot, but you can do it any way you want. One key is that this solution depends on a new feature found in Spring Framework version 4.0.

In my webservice I have a @Controller that has @RequestMappings for GET, PUT, POST, and DELETE, following the normal REST semantics for each method. Now, I have this webservice securely deployed in my production environment and all of my internal services can hit it and everything is awesome.

Now let’s pretend I want to expose some of the resources on the big, bad internet. I want to expose all the GET resources so my front end developers can read the information and put it in a web page, or so my mobile apps can get at it, but I don’t really want to expose the ability for them to create, update, or delete information. Now I’ve got a couple of options.

Option 1

I create a new webservice.  It shares the dependencies of the original so it has access to all the same services, but the controller doesn’t contain any RequestMappings other than the GET resources I want to expose.  This is very secure because I have total control over what is available.  IF the original service was designed appropriately so the Controllers don’t contain any business logic then you can easily reuse all of the logic in the previous webservice.  If not, then it’s a good opportunity to get that done I guess.  On the downside, you now have two artifacts to maintain and deploy.

Option 2

I create a webservice that will proxy requests from the big, bad internet and send them to my internal webservice.  The proxy returns a 404 for any resource/method that should not be exposed, and forwards other requests on to the internal webservice.  Again, my service is secure and I can manage which of the resources are exposed.  Also, again, I have two deployables, and this time they aren’t nearly as related as they were before.  The proxy can be very thin, possibly something as simple as nginx or apache with appropriate rules.

Option 3

This is the option I will explore.  With this option, I modify my webservice so that it can be deployed internally AND externally and lock down the resources that shouldn’t be exposed to the public without having to create a separate deployable artifact.  We will simply annotate those request handlers that should be exposed to the public, basically forming a white-list, and all those that are not explicitly exposed will be restricted from view when certain conditions are met.

In addition, this solution will automatically apply a Jackson JsonView to restrict which properties of the data are exposed, not just which request mappings are exposed.  This will allow us to give a restricted view of the response for the general public on the big bad internet, and the full data for those hitting our internal deployment of the webservice.  We would still be deploying to two environments, one for the public and one for internal, but it would be the same artifact in both places.

First, we are going to use the new @Conditional annotation that was introduced with Spring 4.0.  It allows you to conditionally create a Spring bean.  We will use conditionally defined beans to modify the behavior of the application at runtime.

To The Code

First, the Condition that allows us to change the behavior of the application without having to change any code. My condition is based on the IP address assigned to the server. You could modify the condition to whatever fits your needs. Maybe it checks an environment variable or something. It’s important to note that this condition is evaluated when the bean is created, so if it’s a singleton bean it’ll only be evaluated once. If you are looking to have the condition depend on something from the client then it would probably have to be a request scoped bean, but I haven’t checked to see if that actually works or not. It seems like it should.

/**
 * Condition to check if we are in production or not.
 */
public class ProductionCondition implements Condition {

  @Override
  public boolean matches(ConditionContext context, AnnotatedTypeMetadata meta) {
    Enumeration ifaces;
    try {
      ifaces = NetworkInterface.getNetworkInterfaces();
      while ( ifaces.hasMoreElements()) {
        NetworkInterface iface = ifaces.nextElement();
        Enumeration addresses =  iface.getInetAddresses();
        while ( addresses.hasMoreElements()) {
          InetAddress address = addresses.nextElement();
          // Set whatever your public, production IP Address space is here!
          if ( address.getHostAddress().startsWith("192.168" )) {
            // If we match, then return true so the bean annotated with this conditional will be created.
            return true;
          }
        }
      }
    }
    catch (SocketException e) {
    }
    return false;
  }
}

Now we can use the above Condition to conditionally create Spring beans.

Here’s my Spring Boot application.  It also defines other beans for my spring-data-jpa repositories, but those aren’t relevant to what we are doing so I’ve left them out.

@Configuration
@ComponentScan
@EnableAutoConfiguration
@EnableJpaRepositories
public class Application {

  public static void main (String[] args ) {
    SpringApplication.run(Application.class, args );
  }

  @Configuration
  @Conditional(ProductionCondition.class)
  static class WebConfig extends WebMvcConfigurerAdapter {
    @Override
    public void configureMessageConverters(List<HttpMessageConverter> converters) {
      MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
      ObjectMapper mapper = new ObjectMapper() {
        private static final long serialVersionUID = 1L;
        @Override
        protected DefaultSerializerProvider _serializerProvider(SerializationConfig config) {
          return super._serializerProvider(config.withView(Views.Public.class));
        }
      };
      mapper.configure(MapperFeature.DEFAULT_VIEW_INCLUSION, false);
      converter.setObjectMapper(mapper);
      converters.add(converter);
    }
  }

  /**
   * Only create this bean if we are in "production" mode.
   * @return
   */
  @Bean
  @Conditional(ProductionCondition.class)
  public MappedInterceptor publicHandlerInterceptor() {
    return new MappedInterceptor(null, new PublicHandlerInterceptor());
  }

  // Other beans here for JPA configuration
}

Notice that in the application I have two @Conditional beans. One is a new HandlerInterceptor that I’ll show in a second. The other is a full @Configuration. Because the publicHandlerInterceptor @Bean definition returns a MappedInterceptor it will automatically be configured within the Spring MVC application. If it returned a HandlerInterceptor then more work would have to be done to register it with the Spring MVC application.

Secondly, notice that the Conditional Configuration class extends the WebMvcConfigurerAdapter for allowing me to easily configure Spring MVC-type functionality. Sadly, configuring a custom Jackson ObjectMapper in Spring is much more painful (IMO) than it ought to be, so I’m going to get off on a bit of a tangent. Skip to the next section if you are confident in your ObjectMapper abilities.

ObjectMapper Tangent

It would be fantastic if I could configure the ObjectMapper used for a @ResponseBody by simply defining a @Bean named objectMapper and be good to go. Sadly, that’s not the case. I had to add the MessageConverter in the configuration, and set the ObjectMapper for that MessageConverter. Now, here’s the rub. I kept trying to make my configuration changes to the ObjectMapper by calling getSerializationConfig().blah(). Jackson SerializationConfig is immutable Calling getSerializationConfig() and then all of the handy .with(MappingFeature) or whatever just doesn’t work because it simply returns a new instance of SerializationConfig, but doesn’t modify the one that is in the ObjectMapper. You can see my learning process for this at StackOverflow

Back to the Show

So, the reason I needed to modify the ObjectMapper configuration was so that I could make it always use a given Jackson JsonView for every @ResponseBody encountered. The custom implementation of the ObjectMapper I pasted was the first way I found to configure it to always use the JsonView I specified, otherwise I had to call writeWithView on the writer, and I wasn’t sure where to do that. This configuration gives us the white-list of data properties that should be serialized in each response.

To use it, simply annotate the object returned as your @ResponseBody with the @JsonView annotation from Jackson, something like:

  @JsonView(value={Views.Public.class})
  public String getName() {
    return name;
  }

Securing the RequestMappings

The Application configuration has a conditional bean for a HandlerInterceptor, which looks like this:

public class PublicHandlerInterceptor extends HandlerInterceptorAdapter {
  @Override
  public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
    HandlerMethod method = (HandlerMethod)handler;
    if ( method.getMethodAnnotation(Public.class) != null ) {
      return true;
    }
    response.setStatus(404);
    return false;
  }
}

This HandlerInterceptor will be evaluated for every RequestMapping. Here, we look at the actual method that is being called to handle the request. If it is annotated with our custom @Public annotation, then we allow the request to proceed by returning true from the HandlerInterceptor. If it isn’t, then we return false and send a 404 to the client.

Finally, here’s the Public annotation definition

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface Public {}

And its usage:

  @Public
  @RequestMapping(method=RequestMethod.GET, produces=MediaType.APPLICATION_JSON)
  public @ResponseBody Iterable getCollection(
    @RequestParam(value="ids", required=false) List ids,
    @RequestParam(value="limit", required=false, defaultValue="100") int limit ) {
      // lookup a collection of MyObjects and return them
  }

  @RequestMapping( value="/{id}", method=RequestMethod.PUT, consumes=MediaType.APPLICATION_JSON, produces=MediaType.APPLICATION_JSON)
  public @ResponseBody MyObject putValue(@PathVariable Long id, @RequestBody MyObject d ) {
    // do some things to update an object and return the representation of the updated object
  }

With this in place, I’m able to deploy my webservice (with spring-boot it’s just a jar that contains embedded tomcat!) and run it without any further alterations. The getCollection method would be available in both deployment locations. The putValue handler would only be available in those deployment locations that do NOT match the condition I have specified, so only those that are visible internally. The representation of MyObject is appropriate for the deployment location without any further changes to the webservice either. I merely select the properties of MyObject that I want exposed publicly and annotate them with the appropriate JsonView.

A white-list approach ensures that nothing slips through the cracks to the big, bad internet just because a developer forgot to restrict it. Instead, they must evaluate each request handler and data property and explicitly expose it in the public view.

I could have had my proof of concept developed and tested in under 2 hours had I not run into my difficulties configuring the ObjectMapper. That’s a lesson I won’t soon forget though. I tested all this by making the condition match my IP address when I was connected to my work VPN. When I started the application up and I was connected it would restrict the request handlers and the serialized properties. If I was not connected I could execute any method and would see all of the data properties.

It’s probably not perfect a perfect solution. Does such a thing exist? The one question I’ve thought of is what happens if my code is already using JsonViews? I’m not sure how it would play together. Nevertheless it is an interesting exploration of the capabilities of the @Conditional annotation and HandlerInterceptors.

2011/06/15

Spring ConverterFactory Implementation

Filed under: java, spring — Tags: , , , — digitaljoel @ 10:12 pm

In my Spring MVC 3 based application I had recently implemented a few Converters for some of my JPA based data objects. It started with one, then another, and so on. By the time I got around to adding my fourth converter to the spring configuration file I knew it was time to pull it out and abstract it a bit. Thankfully, Spring allows you to implement a ConverterFactory that is responsible for creating the converters for some types.

Each of my entities extend an abstract base class that looks basically like this

@MappedSuperclass
public abstract class DataObjectAbstract<K extends Serializable>
        implements DataObject<K>
{
    protected transient String[] excludedEqualsFields = new String[] { "key", "version" };

    @Version
    protected int version;

    @Override
    public boolean equals( Object that )
    {
        return EqualsBuilder.reflectionEquals( this, that, excludedEqualsFields );
    }

    @Override
    public int hashCode()
    {
        return HashCodeBuilder.reflectionHashCode( this, excludedEqualsFields );
    }

    @Override
    public String toString()
    {
        return ToStringBuilder.reflectionToString( this, ToStringStyle.MULTI_LINE_STYLE );
    }
}

The DataObject interface simply declares a getKey and setKey method.

So, in my Spring MVC Controller methods I was originally accepting a String or Long, then using my own data access objects to lookup the entities I needed. The next iteration in my implementation was to implement the Converters as I mentioned above. That was very simple and worked well, but having many data objects I didn’t want to copy that implementation over and over again. This is where the ConverterFactory comes in. Here’s my implementation:

@Component
public class DataObjectConverterFactory
        implements ConverterFactory<String, DataObject<Long>>
{
    @PersistenceContext
    EntityManager em;

    @Override
    public <T extends DataObject<Long>> Converter<String, T> getConverter( Class<T> type )
    {
        return new GenericLongKeyedDataObjectConverter<T>( type, em );
    }
}

The ConverterFactory interface is basically as simple as the Converter interface. The Class<T> type parameter to the getConverter method tells us what type we are going to convert to.  One option from here is to have a big nasty if/else statement with a bunch of instanceof methods that create a new Converter.  I thought about doing this and passing in the appropriate data access object and performing the lookup.  That would be only two classes and then I could convert all of my DataObjects, but I didn’t like the idea of a bajillion instanceof statements.  So you can see I implemented a GenericLongKeyedDataObjectConverter which takes the target type and the EntityManager as a parameters.  Here’s the implementation of the generic converter class:

/**
 * A generic converter used for converting from a string representation of an entity key to the DataObject itself.
 *
 * @param <T> The type that is to be converted to.
 */
public class GenericLongKeyedDataObjectConverter<T extends DataObject<Long>>
        implements Converter<String, T>
{
    private Class<T> type;
    private EntityManager em;

    /**
     *
     * @param type An instance of Class for the type being converted to
     * @param em EntityManager used to perform the lookup.
     */
    public GenericLongKeyedDataObjectConverter( Class<T> type, EntityManager em )
    {
        this.type = type;
        this.em = em;
    }

    @Override
    public T convert( String stringKey )
    {
        Long key = Long.parseLong( stringKey );
        return em.find( type, key );
    }
}

An extremely simple parameterized class implementation of the Converter interface. Here, with no use of instanceof, I’m creating the appropriate converter implementation for all of my persisted classes.  If you have a group of objects that you want converted and they all inherit from a base class, a ConverterFactory may be a better solution than implementing a bunch of converters manually.

Finally, here’s the bean xml configuration:

<bean id="conversionService" class="org.springframework.format.support.FormattingConversionServiceFactoryBean">
    <property name="converters">
        <list>
            <ref bean="dataObjectConverterFactory" />
        </list>
    </property>
</bean>

Notice that we reference the dataObjectConverterFactory bean, but I never defined it in my xml config.  That’s because I used the @Component annotation on my implementation class.

2011/02/05

Using Mockito To Test Spring MVC Ajax Interaction

Filed under: java, spring, testing — Tags: , , , , , , , — digitaljoel @ 4:36 pm

So, I shared in Ajax Post to Spring MVC Controller what I learned about making an ajax post to a Spring MVC Controller. Then I shared in Mock Testing Spring MVC Controller what I learned about using Mockito to test my Spring MVC controller. So what about testing my RequestHandler that handles the ajax post and returns a JSON object? Well, as Samuel L. Jackson says in Jurassic Park, “Hold on to your butts”

Here’s the method that handles the ajax post of form data.

    @RequestMapping( value="answer/new", method=RequestMethod.POST)
    public ResponseEntity<String> newAnswer( @RequestParam(value="answerSeverity", required=true ) String severity
            , @RequestParam( value="answerText", required=true ) String text
            , @RequestParam( value="requiresReason", required=false, defaultValue="false" ) boolean requiresReason
            , @RequestParam( value="answerReasons", required=false ) List<Long> reasonKeys
            )
    {
        Severity sev = Severity.valueOf( severity );
        SurveyAnswer answer = new SurveyAnswer( text, sev );
        answer.setRequiresReason( requiresReason );
        if ( requiresReason )
        {
            // add all the reasons
            List<SurveyAnswerReason> reasons = surveyService.findReasonsByKey( reasonKeys );
            for( SurveyAnswerReason reason : reasons )
            {
                answer.addReason( reason );
            }
        }
        answer = surveyService.persist( answer );
        this.getAnswers( sev ).add( answer );
        return createJsonResponse( answer );
    }

    private ResponseEntity<String> createJsonResponse( Object o )
    {
        HttpHeaders headers = new HttpHeaders();
        headers.set(  "Content-Type", "application/json" );
        String json = gson.toJson( o );
        return new ResponseEntity<String>( json, headers, HttpStatus.CREATED );
    }

You can read the previous post for information on what’s going on here, but basically, we handle the form post, create a new SurveyAnswer, and then return the created answer as a JSON object using the createJsonResponse method.

In order to mock test this, I’m going to have to mock all the calls to the surveyService methods. That would be findReasonsByKey, and persist. The persist was a bit tricky because I wanted it to just return the answer that was passed as an argument to ensure that the controller was creating the answer correctly. Here’s the code to do that.


        when( surveyService.persist( any( SurveyAnswer.class ))).thenAnswer(
                new Answer<SurveyAnswer>()
                {
                    @Override
                    public SurveyAnswer answer( InvocationOnMock invocation ) throws Throwable
                    {
                        Object[] args = invocation.getArguments();
                        return (SurveyAnswer) args[0];
                    }
                });
        when ( surveyService.findReasonsByKey( anyCollectionOf( Long.class ))).thenReturn( getReasons() );

I put it in my @Before annotated setup method in my unit test. I didn’t come up with it myself, I adapted it from an excellent answer to this question on StackOverflow.com. That snippet allows me to just return the argument passed to the method, which is basically what JPA would do, other than setting the key and version, which I don’t really need for my test anyway. The mocked out findReasonsByKey just returns a list of objects that I’m creating elsewhere for testing purposes only.

So, on to the test. Here’s the code:


    @Test
    public void testNewAnswerWithReasons()
    {
        ResponseEntity<String> response = controller.newAnswer( answerSeverity.name(), answerText,
                true, getReasonKeys() );
        assertEquals( "application/json", response.getHeaders().get( "Content-Type" ).get( 0 ));
        SurveyAnswer answer = gson.fromJson( response.getBody(), SurveyAnswer.class );
        assertEquals( getSingleAnswerWithReasons(), answer );
    }

There are some helper methods that create the object graph needed for the SurveyAnswer. It then calls the method on the controller (which is also initialized in the @Before setup method) and checks the result. I’m really looking for two things. First, that the response has the Content-Type set correctly to application/json, and second, that I get an answer that corresponds to the values I passed in. Here again, I use Google’s GSON library for converting from my JSON string to my Java object. Once that is done, I can just test for equality with the answer I’m expecting. Obviously, for that to work, you’ll need to make sure your equals method is correct, but that’s an issue well addressed elsewhere on the internet and well beyond the scope of this post.

Mock Testing Spring MVC Controller

Filed under: development, spring, testing — Tags: , , , , , , — digitaljoel @ 2:08 pm

I’m in the midst of implementing a Spring MVC based web application. We have hudson for continuous integration builds that run all of our unit tests, but testing spring MVC controllers still just isn’t quite as easy as I would hope. There is some information on testing the controllers in the official spring documentation, but for someone like me that’s not a spring guru, or just starting out, it wasn’t enough to get me going. I recently was introduced to Mockito, so I spent a bit of time today trying to get a test for our controller using Mockito. It was simple and took no time at all.

I have yet to try it in a more complex controller method, but I think it’ll work just fine , especially when I get some utilities in place to initialize the mock objects that are commonly used by the controller. As it stands, here’s what I did to get it going. This tests the controller as a POJO, without using any spring configuration or capabilities.

Here is the controller I wish to test. Obviously I stripped out a bunch of code not needed for this demonstration.

@Controller
@RequestMapping( "/admin/survey" )
public class SurveyAdminController
{
    
    @Resource
    private SurveyService surveyService;
    
    @Resource
    private UnitService unitService;
    
    @Resource
    private OrganizationService orgService;
    
    /**
     * Show the table of existing questions to the user
     * @param model
     * @return
     */
    @RequestMapping( "questions" )
    public ModelAndView listQuestions()
    {
        ModelAndView mav = new ModelAndView( "/admin/questionList");
        List<SurveyQuestion> questions = surveyService.findAllQuestions();
        mav.addObject( "questions", questions );
        List<UnitFeature> features = unitService.getAllFeatures();
        mav.addObject( "features", features );
        List<Organization> organizations = orgService.getAll();
        mav.addObject( "organizations", organizations );
        return mav;
    }
}

For this simple test, it’s just going to validate that the view name is correct, and that the model that’s returned contains the correct information.


package com.bi.controller;

import static org.junit.Assert.assertEquals;
import static org.mockito.Mockito.when;

import java.util.ArrayList;
import java.util.List;

import org.junit.Before;
import org.junit.Test;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;
import org.springframework.test.util.ReflectionTestUtils;
import org.springframework.web.servlet.ModelAndView;

import com.bi.data.OrganizationService;
import com.bi.data.SurveyService;
import com.bi.data.UnitService;
import com.bi.data.corp.Organization;
import com.bi.data.survey.SurveyQuestion;
import com.bi.data.unit.UnitFeature;
import com.bi.web.util.MessageUtil;

public class SurveyAdminControllerTest
{
    @Mock SurveyService surveyService;
    @Mock UnitService unitService;
    @Mock OrganizationService orgService;
    @Mock MessageUtil messageUtil;

    @Before
    public void setup()
    {
        // this must be called for the @Mock annotations above to be processed.
        MockitoAnnotations.initMocks( this );
    }
    
    @Test
    public void testListQuestions()
    {
        // setup our mock question list
        List<SurveyQuestion> questions = new ArrayList<SurveyQuestion>();
        questions.add( new SurveyQuestion( "asdf", null ));
        when( surveyService.findAllQuestions()).thenReturn( questions );

        // setup our mock feature list
        List<UnitFeature> features = new ArrayList<UnitFeature>();
        features.add( new UnitFeature( "TEST FEATURE" ));
        when( unitService.getAllFeatures()).thenReturn( features );

        // setup our mock organization list
        List<Organization> orgs = new ArrayList<Organization>();
        orgs.add( new Organization( "TEST ORGANIZATION" ));
        when( orgService.getAll()).thenReturn( orgs );

        // create an instance of the controller we want to test
        SurveyAdminController controller = new SurveyAdminController();

        // since we aren't using spring, these values won't be injected, so set them manually
        ReflectionTestUtils.setField( controller, "surveyService", surveyService );
        ReflectionTestUtils.setField( controller, "unitService", unitService );
        ReflectionTestUtils.setField( controller, "orgService", orgService );

        // call the method under test
        ModelAndView mav = controller.listQuestions();

        // review the results.
        assertEquals( questions, mav.getModel().get( "questions" ));
        assertEquals( features, mav.getModel().get( "features" ));
        assertEquals( orgs, mav.getModel().get( "organizations" ));
        assertEquals( "/admin/questionList", mav.getViewName());
    }
}

The comments should be pretty self explanatory in the test class. The awesomeness of Mockito is how you setup the mocks. A line like this:

        when( unitService.getAllFeatures()).thenReturn( features );

reads very nicely and sets up the return value for my service object, which keeps me from needing a database or anything setup in order to test my controller method. Mockito ftw.

2011/01/29

Ajax Post to Spring MVC Controller

Filed under: development, java, jquery, spring — Tags: , , , , , — digitaljoel @ 7:38 pm

I wanted to submit an html form to my Spring MVC Controller, but I wanted to do it with ajax. I had previously submitted a single value and returned a JSON object for use in jquery, but I had yet to do it with an entire form. I’m a java guy, so there may be better ways to do the html stuff, but this is how I did it and thought I would share some of what I learned along the way.

First, here’s my form.

    <div id='newAnswerDialog'>
        <form id='newAnswerForm' name='newAnswerForm' action='/admin/survey/answer/new' onsubmit='return false;' method='post'>
            <label for='severity'><spring:message code='input.answer.severity' text='Severity' /></label>
            <select id='answerSeverity' name='answerSeverity'>
                <option value='MINIMAL'>MINIMAL</option>
                <option value='MINOR'>MINOR</option>
                <option value='MODERATE'>MODERATE</option>
                <option value='SEVERE'>SEVERE</option>
                <option value='URGENT'>URGENT</option>
            </select><br/>
            <label for='answerText'><spring:message code='input.answer.text' text='Answer Text' /></label>
            <input type='text' id='answerText' name='answerText'/><br/>
            <label for='requiresReason'><spring:message code='input.answer.requiresReason' text='Requires Reason?' /></label>
            <input type='checkbox' id='requiresReason' name='requiresReason' onclick='toggleReasonControls(this)'/><br/>
            <label for='answerReasons'><spring:message code='input.question.reasons' text='Reasons' /></label>
            <select class='newReason' id='answerReasons' name='answerReasons' multiple='multiple'>
                <c:forEach items='${reasons}' var='reason'>
                    <option value='${reason.key}'>${reason.text}</option>
                </c:forEach>
            </select><br/>
            <input type='button' onclick='createNewAnswer()' value='<spring:message code='submit' />'/><input type='button' onclick='cancelNewAnswer()' value='<spring:message code='cancel' />' />
        </form>
    </div>

It’s JSP using the JSTL and the Spring tag libraries, but it’s all basically html. You can see that I have some javascript in the requriesReason checkbox that is called when it is clicked. The javascript function just enables everything with a css class of newReason based on the state of the requiresReason checkbox. This will come into play later.

When the user clicks the button labeled submit, it calls the javascript function called “createNewAnswer()” which looks like this


        function createNewAnswer()
        {
            $.post( '<c:url value='/admin/survey/answer/new' />'
                    , $('#newAnswerForm').serialize()
                    , function( data )
                    {
                        // add the option to the list of answers, and select it.
                        var options = $('#'+data.severity.toLowerCase()+'Answer').attr( 'options' );
                        options[options.length] = new Option( data.text, data.key, true, true );
                        $('#newAnswerDialog').dialog( 'close' );
                    }
                    , 'json' );
        }

This uses the jquery post function to submit the values to the server and read the response.

The first parameter is the URL to submit to. I’m using the JSTL c:url tag so that the web application context is added to the url. Your controller method must be configured to accept POST requests since we are submitting via POST, not GET. I’ll show the controller implementation later on.

The second parameter is the data you want to submit to the Controller. This is where I found it a bit tricky. Getting the form is simple with jquery. Getting the data is also simple using jquery’s serialize() method. The trick is in the fine print in the serialize method. You can find the documentation here. At the time of this writing, it says:

Note: Only “successful controls” are serialized to the string. No submit button value is serialized since the form was not submitted using a button. For a form element’s value to be included in the serialized string, the element must have a name attribute. Data from file select elements is not serialized.

It then links to this page that has an explanation of what a “successful control” is. The gist of it is that the control cannot be disabled. The control must have a name. Checkboxes that are not checked may not be submitted. At least, those are the parts that affected me when trying to get this to work.

So, knowing that, it affects how I create the controller method to handle the post. Here’s the implementation:

    @RequestMapping( value='answer/new', method=RequestMethod.POST)
    public ResponseEntity<String> newAnswer( @RequestParam(value='answerSeverity', required=true ) String severity
            , @RequestParam( value='answerText', required=true ) String text
            , @RequestParam( value='requiresReason', required=false, defaultValue='false' ) boolean requiresReason
            , @RequestParam( value='answerReasons', required=false ) List<Long> reasonKeys
            )
    {
        Severity sev = Severity.valueOf( severity );
        SurveyAnswer answer = new SurveyAnswer( text, sev );
        answer.setRequiresReason( requiresReason );
        if ( requiresReason )
        {
            // add all the reasons
            List<SurveyAnswerReason> reasons = surveyService.findReasonsByKey( reasonKeys );
            for( SurveyAnswerReason reason : reasons )
            {
                answer.addReason( reason );
            }
        }
        answer = surveyService.persist( answer );
        return createJsonResponse( answer );
    }

Notice that the requiresReason, and answerReasons are marked as optional. This is because they may or may not be passed. If you have a problem with your mapping here, you may get a 400 error with a message that says, “The request sent by the client was syntactically incorrect ()” if you look at it in the XHR response. That’s what I was getting from tomcat. Once I set the required to false on the optional attributes, things went through much better.

The rest of the code is just for creating our entity and saving it. You can do whatever you want with the data you post. The next critical part is in the createJsonResponse method. You’ll notice that the controller method doesn’t return a String for the view, and it doesn’t return a ModelAndView. It is returning a ResponseEntity, which I create in the createJsonResponse method, which is as follows:


    private ResponseEntity<String> createJsonResponse( Object o )
    {
        HttpHeaders headers = new HttpHeaders();
        headers.set(  'Content-Type', 'application/json' );
        String json = gson.toJson( o );
        return new ResponseEntity<String>( json, headers, HttpStatus.CREATED );
    }

It’s a very simple method that just creates the ResponseEntity and sets the headers so the javascript receiving the response gets json as it is expecting (see the last parameter in the $.post method, it says we are expecting json in return.)

I am using Google’s JSON library to convert my entity object to a JSON object that I can return to the view. With that conversion, I can use the field names in the java object to reference the values in the returned JSON object. It’s really quite slick.

So, now we’ve done the work on the server and returned the response to the browser. The third argument to the $.post jquery method is a function that is called on successful return of the json object from the server. Here is my function again:

                    function( data )
                    {
                        // add the option to the list of answers, and select it.
                        var options = $('#'+data.severity.toLowerCase()+'Answer').attr( 'options' );
                        options[options.length] = new Option( data.text, data.key, true, true );
                        $('#newAnswerDialog').dialog( 'close' );
                    }

I’m taking the JSON object I receive and adding a new option to a select box further down in the page, and marking that option as selected. As I said above, I’m referencing the properties of the JSON object using the field names in the Java object.

There is a way to get a javascript method called when the request fails, but it’s not built into jquery’s post method, and I haven’t taken the time go through that part yet.

So, there you have it. A fair amount of time of trial and error all summed up in about 1000 words, including code.

2011/01/15

Simple Application Configuration With Spring

Filed under: development, java, spring — Tags: , , — digitaljoel @ 1:48 am

Inevitably it happens in every project. You’re coding along, and you have a “constant” that isn’t really constant. In my case, it was the number of minutes that should pass before a given token is invalid. 2 hours? Yeah, that sounds good… for now. But you feel all dirty if you put it right in a .java file. You could provide the value as a property to your bean if you are doing XML configuration, but is that really that much better? You want it in a properties file, because that’s one reason properties files exist, and it’s all configurable and stuff, right?

The company at which I spend my daylight hours has a ConfigurationService that takes care of hierarchical property overriding and all that. I’m sure it’s nifty. I didn’t want to write anything like that for the project on which I spend my evening/weekend hours, so I found something in spring that would do it for me and had it going pretty quickly.

Alright, here we go with the simple solution built right into Spring.

My project is a multi-module maven project with a single parent pom and each module under that parent project. I believe it’s a fairly common setup. In a module early in the dependency tree, I created a spring configuration file that looks like this.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

    <!-- a bean for storing configuration properties. -->    
    <bean id="runtimeConfig" class="org.springframework.beans.factory.config.PropertiesFactoryBean">
        <property name="locations">
            <list>
                <value>classpath*:/META-INF/config/*.properties</value>
                <value>file:///${user.home}/myproject.properties</value>
            </list>
        </property>
    </bean>
    
</beans>

Yep, one bean. It creates a single java Properties object from the list of files I give it, from top to bottom, with properties in later files overriding properties in earlier files. In this case, I added the wildcard /META-INF/config/*.properties, so all of my other modules can put their default property values in properties files in META-INF/config and if they depend on this module, it’ll find them. Note that you must use classpath*: and the path must have at least one directory in it or else you will likely read from the location in only one of your jar files.

Some considerations here:

If you are using another jar, say from a third party vendor, and they have properties in META-INF/config, then you may load all those properties too. I haven’t seen it in my limited testing, but that would stink. You might be better off changing that to something like META-INF/config/myproject-*-default.properties, then you would be more sure that you will only get your properties. You don’t really want to put specific properties file names in here because you don’t want this module, which is early in the dependency tree, to know explicitly about properties files in modules it knows nothing about.

Also, depending on how your properties files within your jar files are named, your property overriding may not occur as you hoped. Hopefully, you don’t have properties with the same name in different properties files, in which case this is a non-issue. If you do have properties with the same name in different properties files in your various modules you may have to be even more careful in how they are loaded in your bean definition.

Finally, with the file:///${user.home}/myproject.properties value last, you can override any of the properties in the default configuration files with properties in a single file in your user home directory. This is pretty cool. This way your application can define the default timeout at 120 minutes but when you are testing and don’t want to wait 2 hours for a timeout you just set the value in your home directory configuration file to 1 minute and Bob’s your uncle. No risk of checking in that configuration value because it’s not under source control.

Another thing you could likely do here is to load different properties files based on some environment variable. You want to test against postgres? Set an environment property when you run your build and set the pattern to something like file:///${user.home}/${dbtype}-config.properties. Each ${dbtype}-config.properties file could give connection information for the specified database, so your home directory would have postgres-config.properties, oracle-config.properties, mysql-config.properties, etc. Then, in hudson, bamboo, or whatever you define a property for the build that has the database type in it. For maven command line, your command would be something like this. I tried it with a simple property and it worked great.

mvn -DargLine="-Ddbtype=postgres" test

Ok, moving on. Now we want to use this fancy properties object we’ve created.

So, we’ve moved down the dependency tree to a module that depends on my module with the fancy runtimeConfig bean. In my spring configuration within that module, I do the following:

<!-- all that other spring stuff you have defined -->


    <import resource="classpath:/META-INF/spring/domain-api-beans.xml"/>
    <context:property-placeholder properties-ref="runtimeConfig" />

    <bean id="yourDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
        <property name="driverClassName" value="${jdbc.driverClassName}"/>
        <property name="url" value="${jdbc.url}"/>
        <property name="username" value="${jdbc.username}"/>
        <property name="password" value="${jdbc.password}"/>
    </bean>

So, we import our previous spring configuration file so we can use the awesome runtimeConfig bean. Next, we create a property-placeholder so we can reference the properties in our runtimeConfig within this very same configuration file. Then, magically, all of the ${…} values in our datasource definition are replaced by the values in our properties files.

“BUT WAIT!” You say, “This isn’t the use case you mentioned at the top of the post. I can do this by specifying the locations of the property-placeholder.” Yes. Yes you can. This was just extra credit to show that we don’t have to point to the same locations in two definitions. Just once will do.

So, now we have a service or some bean or something, and we want to get our expiration configuration value out of it. How would we accomplish such a thing? Funnily enough, that’s the easiest part of all this.


    // all my service bean code or whatever

    // here we read the value from the runtimeConfig bean using Spring's expression language.
    @Value("#{runtimeConfig['account.validation.expiration.minutes']}" )
    private long minutes;

    // here's a nice method to return the value so I don't have that ugly @Value thing everywhere I want to reference the value.
    public long getValidityMinutes()
    {
        return minutes;
    }
    
    // the rest of my service bean code and stuff

Voila, I’ve successfully read the property from the file in my user home, and if it’s not set there, then I’ve read it from the reasonable defaults set within the properties file in the jar that uses the value. I didn’t have to write any service, bean, or anything to implement a hierarchical properties based configuration system for my application.

Update: One thing I failed to mention, which you probably already figured out, is that since the runtimeConfig is just a bean, if you know that you are going to be reading a ton of properties out of it, you could just inject the entire bean, something like this.

@Resource( "runtimeConfig" )
private Properties config;

2010/12/28

Spring MVC and JSR-303 Validation Groups

Filed under: development, java, spring — Tags: , , , , , , — digitaljoel @ 8:27 pm

@Valid. That wonderful little annotation. In my Spring controller I do something like this.

    @RequestMapping( value="/editAccount", method=RequestMethod.POST )
    public String postEditAccount( Model model, HttpServletRequest request, 
            @Valid AccountInfo info, BindingResult infoResult )

and everything is great. I know that all of the JSR-303 annotations I’ve put on my AccountInfo bean have been validated and the data is all correct and good. Let’s say that my AccountInfo bean looks something like this.

public class AccountInfo
{

    @NotNull
    private String username;
    @NotNull
    private String newUsername;
    @NotNull
    private String confirmNewUsername;
    
    @NotNull
    private String password;
    @NotNull
    private String newPassword;
    @NotNull
    private String confirmNewPassword;
    
    @NotNull
    private String firstName;
    @NotNull
    private String lastName;
    @NotNull
    private String phone;
    private String fax;
    
    // constructors, getters, setters, etc. down here
}

Suddenly, everything isn’t so hunky dory anymore. Fortunately for us, JSR-303 has a great mechanism for only validating some portion of the object. It’s known as validation groups. There’s plenty of information out there on them, so I’ll give you the extreme Reader’s Digest version. Basically, you specify a list of marker interfaces in your validation annotations, and then when you call the validator, you can also pass in a list of the marker interfaces that you would like to validate against. Ok, that sentence doesn’t make much sense unless you already know about groups. So, here’s a new version of AccountInfo that demonstrates.


public class AccountInfo
{

    @NotNull( groups={ChangeUsername.class} )
    private String username;
    @NotNull( groups={ChangeUsername.class} )
    private String newUsername;
    @NotNull( groups={ChangeUsername.class} )
    private String confirmNewUsername;
    
    @NotNull( groups={ChangePassword.class} )
    private String password;
    @NotNull( groups={ChangePassword.class} )
    private String newPassword;
    @NotNull( groups={ChangePassword.class} )
    private String confirmNewPassword;
    
    @NotNull
    private String firstName;
    @NotNull
    private String lastName;
    @NotNull
    private String phone;
    private String fax;
    
    // constructors, getters, setters, etc. down here
    
    public interface ChangePassword {};
    public interface ChangeUsername {};

}

Now we’ve told the validator that when we run the validator without any groups, we want it to validate firstName, lastName, and phone. If you don’t specify any groups, they get the Default.class group. BUT, if we run the validator passing in the AccountInfo.ChangePassword.class, then it will only validate password, newPassword, and confirmNewPassword. If we want to do both, then we can pass in AccountInfo.ChangePassword.class AND Default.class and it will validate both groups. That’s awesome sauce right there. Now, we can use this same backing bean in the page where the user is created which contains all the fields, we can use it in the edit account info page which only has the stuff validated by Default, we can use it in our change password page, and we can also use it in our change username page, and in each case, we only validate the portions that we are concerned about for those pages. One bean for all four pages.

With that worked out, we should be able to just add the groups to our @Valid annotation, right? Nope. Wait, what? All that work to put in validation groups and we can’t even use them with the JSR-303 sanctioned validation annotation? Yep, that’s right. There’s an improvement in Spring’s Jira to add a new @Valid annotation that will allow you to specify groups, but until that happens, you’ll have to run the validator yourself.

I think it sound worse than it is. As you can see in the controller method I put at the start of this post, that after each @Valid annotated object, you need to have the BindingResult in order to see the errors. Then, in your controller method you have to check the BindingResult in order to see if there are errors, if there are, do something, if not, do something else. So, how is that any different than having to just run the check yourself? Here’s what I did.


    /**
     * Test validity of an object against some number of validation groups, or
     * Default if no groups are specified.
     * 
     * @param result Errors object for holding validation errors for use in
     *            Spring form taglib. Any violations encountered will be added
     *            to this errors object.
     * @param o Object to be validated
     * @param classes Validation groups to be used in validation
     * @return true if the object is valid, false otherwise.
     */
    private boolean isValid( Errors result, Object o, Class<?>... classes )
    {
        if ( classes == null || classes.length == 0 || classes[0] == null )
        {
            classes = new Class<?>[] { Default.class };
        }
        Validator validator = Validation.buildDefaultValidatorFactory().getValidator();
        Set<ConstraintViolation<Object>> violations = validator.validate( o, classes );
        for ( ConstraintViolation<Object> v : violations )
        {
            Path path = v.getPropertyPath();
            String propertyName = "";
            if ( path != null )
            {
                for ( Node n : path )
                {
                    propertyName += n.getName() + ".";
                }
                propertyName = propertyName.substring( 0, propertyName.length()-1 );
            }
            String constraintName = v.getConstraintDescriptor().getAnnotation().annotationType().getSimpleName();
            if ( propertyName == null || "".equals(  propertyName  ))
            {
                result.reject( constraintName, v.getMessage());
            }
            else
            {
                result.rejectValue( propertyName, constraintName, v.getMessage() );
            }
        }
        return violations.size() == 0;
    }

Alright, it’s a pretty simple method, but we’ll walk through it.


        Validator validator = Validation.buildDefaultValidatorFactory().getValidator();
        Set<ConstraintViolation<Object>> violations = validator.validate( o, classes );

Here we get the validator instance and get the set of violations back. This is based on the validation groups that were passed in, or Default if there were none passed. I believe I read somewhere that you can actually get the Validator injected by Spring, but I haven’t played with it yet to find out. If you do and it works, let me know!

Next is the part where we take the JSR-303 validations and map them to Spring form errors.


            Path path = v.getPropertyPath();
            String propertyName = "";
            if ( path != null )
            {
                for ( Node n : path )
                {
                    propertyName += n.getName() + ".";
                }
                propertyName = propertyName.substring( 0, propertyName.length()-1 );
            }

We get the property name of the violation, which will hopefully map to the “path” in the spring input tag you are using. I haven’t tested this on anything with any depth (for instance, if your bean contains an object that also has validation annotations on it) so I’m not sure how it’ll work there. Once again, if you find out, leave a comment. Anyway, now that we have the property name, we can use that later on to tell Spring which field failed validation so the correct errors field can be shown.


            String constraintName = v.getConstraintDescriptor().getAnnotation().annotationType().getSimpleName();

Now we get the name of the constraint that failed. In all cases above, it would be NotNull. If you annotation is something like @Size( min=85 ) then the constraint would be Size. We use this so we can get error messages mapped the same way spring binding violations do, so if you are using custom messages in your messageSource for your fields or constraint messages, then this should work just the same.


            if ( propertyName == null || "".equals(  propertyName  ))
            {
                result.reject( constraintName, v.getMessage());
            }
            else
            {
                result.rejectValue( propertyName, constraintName, v.getMessage() );
            }

Finally, before returning the true or false, we have to add the violations to the Errors object. If we have a propertyName, then that means it’s a field error. If we don’t, it’s a global object error, which ought to happen if you use a class level validation annotation instead of a field level validation annotation. Yep, I’m going to say it one more time. I haven’t tested that yet, but it won’t be long before I do since I’ve got to get a class level validator to ensure the newPassword and confirmNewPassword fields contain the same value.

So, that’s a lot of explaining for a single method, but hopefully it shows that adding this validation really isn’t that much more difficult than checking the BindingResult that Spring gives back to you after the default @Valid processing. In places where I don’t have to use validation groups, I’m obviously still going to use the @Valid annotation as is, and if Spring gives me a new annotation I can use to run validation groups, I’ll jump to that and rip this out right away. BUT, until then, this will have to do.

Finally, here’s how I call that method in my controller.


    @RequestMapping( value="/editAccount", method=RequestMethod.POST )
    public String postEditAccount( Model model, HttpServletRequest request, 
            AccountInfo info, BindingResult infoResult )
    {
        if ( !isValid( infoResult, info, AccountInfo.ChangePassword.class, AccountInfo.ChangeUsername.class ))
        {
            return "editAccount";
        }
        // otherwise we process the form and do stuff.
    }

Obviously, you would want to move the isValid method out of this controller and into something that can be shared between Controllers.

2010/11/15

Spring MVC binding to Strings and Dates

Filed under: development, spring — Tags: , , , — digitaljoel @ 2:29 pm

I spent the entire morning trying to figure out how to get Spring MVC to allow for Null in my Date field.  I would get an exception if the Date value in the form was left Null.  Once I found that out, I wasn’t getting any validation messages for all the fields marked as @NotNull.  It turns out Spring just set the value to empty string instead of null if the field was empty.  Hibernate’s JSR-303 implementation has a @NotEmpty validation, but I decided to try to keep it to spec.

So, I implemented a custom @InitBinder for my @Controller and had an anonymous implementation of a custom editor all based off an answer on stackoverflow.com.  Finally, I found this bug logged against Roo

https://jira.springsource.org/browse/ROO-190

Using that single line in my @InitBinder method I was then able to set Dates to null.  For the second problem, I used this very helpful blog post by Stefan Reuter

http://blogs.reucon.com/srt/2007/11/25/spring_mvc_null_or_an_empty_string.html

So, now my @InitBinder method looks like this.

    @InitBinder
    public void allowEmptyDateBinding( WebDataBinder binder )
    {
        // Allow for null values in date fields.
        binder.registerCustomEditor( Date.class, new CustomDateEditor( new SimpleDateFormat( getDatePattern()), true ));
        // tell spring to set empty values as null instead of empty string.
        binder.registerCustomEditor( String.class, new StringTrimmerEditor( true ));
    }

And as simple as that I get null instead of empty string for my string values, and I can allow null values in my non-required date fields. Too bad it took me 6 hours this morning to find the answers.

Blog at WordPress.com.