Spring Boot JWT

In my earlier posts, I described how to do authentication and authorization for a custom database schema - with Spring Security and PicketLink. 

In this post, I use the same custom schema to show how to use JWT with Spring Boot.

Most of these are covered in https://medium.com/wolox/securing-applications-with-jwt-spring-boot-da24d3d98f83

I build on top of this by tweaking it to use the same schema as used in my earlier post where the password is kept in a separate table:

A simple one, with a table to store the user info, a separate related table to store password (this is useful because, when bringing back user from DB, we don't need to bring back the password related info). A master table for role and another table to associate the roles for a user.

I write corresponding JPA entities for these tables. Importantly, I don't have a relation from AppUser to UserPassword entity (it's the other way around) - simply because, as mentioned above, I don't want to bring back password information when I load user. Same applies in case of roles too. One of the basic concepts to remember about security is that, information should not be provided unless and until it is asked for (strictly on a need-to-know basis).

So, how do we encrypt the password and store the same in DB? Spring provides excellent support for encoders. So, we declare a bean in the config file, such as:

@Bean
public PasswordEncoder passwordEncoder() {
  return new BCryptPasswordEncoder();
}

Let us now look at how to create an user with password and associate a role. We can create the entities and persist with Spring Data JPA repository:

String role="ADMIN";

AppUser appUser = new AppUser();

appUser.setLoginName(email);

...

appUserRepository.save(appUser);

UserRole userRole = new UserRole();

userRole.setAppUser(appUser);

userRole.setRole(roleMasterRepository.findByName(role));

userRoleRepository.save(userRole);

UserPassword userPassword = new UserPassword();

userPassword.setAppUser(appUser);

userPassword.setPasswordHash(passwordEncoder.encode(password));

userPasswordRepository.save(userPassword);

The above block of code can be placed within a transaction and we can call this service from wherever user creation needs to be done.

For the user authentication to work, we need to wire a authentication provider which can be  a DaoAuthenticationProvider as we are storing the info in a database. We need to pass an service to this provider which implements org.springframework.security.core.userdetails.UserDetailsService. 

We also need to set the password encoder which is same as we saw above. The UserDetailsService is an interface, so we need to override the method loadUserByUsername() which should return a org.springframework.security.core.userdetails.UserDetails instance. 

So, we create a class which implements UserDetails interface. I have named this as UserLoginDto and I pass the AppUser object and the role of the user. Using this the overridden methods can return corresponding values. For example, the method isEnabled() can simply return appUser.isActive().

Most importantly, we have to override the getAuthorities() method which needs to return the list of roles assigned for the user. Since we have retrieved the roles for the user from the DB, we can use that to build the Set and return it 

(note: I have used only only role per user in this example, we can easily expand on that).

public class UserLoginDto implements UserDetails {

  public UserLoginDto(AppUser appUser, String passwordHash, AppRole appRole) {

    this.appUser = appUser;

    ...

   }

  @Override

  public Collection<? extends GrantedAuthority> getAuthorities() {

    Set<RoleGrantedAuthority> roleSet = new HashSet<>();

    roleSet.add(new RoleGrantedAuthority("ROLE_"+ appRole.toString()));

    return roleSet;

  }

  @Override

  public boolean isEnabled() {

    return appUser.isActive();

  }

  ...

}

We do the configuring for authentication manager, password encoder all in the class WebSecurityConfig (the one annotated with @EnableWebSecurity).

As we are building this for HTTP RESTFul services, we need to have an endpoint to which users will make a REST call and get back a JWT token. And then, we will need to have a filter which will intercept all REST calls and check if the token is valid or not.

So, for the first part, we can simply write a filter which extends org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter.

We need to override two methods: attemptAuthentication, successfulAuthentication

The first method is called when authenticating the user. But, as explained above, we are authenticating via an endpoint. So, what is the endpoint?

As we're extending the Spring's internal UsernamePasswordAuthenticationFilter, by default, Spring sets the endpoint as '/login'. We can change this endpoint to something else with a call to (in the constructor):

super.setFilterProcessesUrl("/authenticate"); 

So, here, I have set the auth endpoint to 'authenticate'. When the user hits the endpoint, as a POST request with body like:

{ "username": "user1", "password": "pwd1"}

Spring Spring calls our attemptAuthentication method. We need to first read the body from the request (JSON parse). And then we call the getAuthenticationManager().authenticate. This will in turn call our authentication manager (which we configured earlier) which uses our UserService (which makes JPA calls to our custom database to load the user along with the password hash).

If the auth is successful, Spring calls our  successfulAuthentication method. It is in this method where we need to create the JWT token.

We're making use of the jjwt library to create the JWT token. We set the login name as the subject in the token. We are also setting the role that the user is assigned to in the claims part. This role information is needed and useful for the caller. We're also setting the expiration time for the token.

We are reading the key hash needed for the JWT token from the properties file and are using it. Likewise, the expiration time is also read from properties file (set in minutes). In the response header, we set the token. Once the user gets back the token, they can use it in subsequent requests. This has to be set in the request header 'Authorization' with the format 'Bearer <token>'.

For subsequent requests, we need to write a filter which should check the token validity. Allowing/disallowing an URL and also allowing specific calls for specific roles are all set in the usual manner in the WebSecurityConfig (this is standard Spring Security stuff).

For URL based authorization, we can set it up in the security config,such as:

.antMatchers("/unsecured/*").permitAll()

.antMatchers("/general/*").authenticated()

.antMatchers("/normal/*").hasRole(AppRole.USER.toString())

.antMatchers("/admin/*").hasRole(AppRole.ADMIN.toString())

I have named the filter for token verification as, well, TokenVerificationFilter. This class extends the Spring org.springframework.security.web.authentication.www.BasicAuthenticationFilter class which extends the org.springframework.web.filter.OncePerRequestFilter.

We need to override the doFilterInternal method. We read the token from the header here and call the parseClaims method of the jjwt library. This call verifies the signature and also checks if the token has expired (by checking the expiration time). If the token is valid, we set the user details in the Spring security context, like:

SecurityContextHolder.getContext().setAuthentication(authentication);

We set these two filters in the WebSecurityConfig such as:

.addFilter(new AuthenticationFilter(keyHash, expMinutes,  authenticationManager()))

.addFilter(new TokenVerificationFilter(keyHash, authenticationManager()))

The complete source code for this post is available in my github repo

References: 

https://www.toptal.com/spring/spring-security-tutorial

https://medium.com/wolox/securing-applications-with-jwt-spring-boot-da24d3d98f83

Nifi Simple Example for Wait Notify

In this post, I try to provide a simple example to demonstrate how wait/notify works in nifi. We're going to look at a scenario, where there is a list of files to be processed. But, we want Nifi to process the files one at a time (or say max n at a time) due to downstream performance restrictions. We're going to use 'GenerateFlowFile' to generate the scenario where there are lot of files to be processed.

We then connect this directly to a 'Wait' processor. The wait processor waits for the signal from a corresponding processor. When it recieves the signal, it proceeds via the 'success' relationship. So, we first 'route' the expired/error relationship to a simple 'LogMessage' processor which just logs a message. The 'wait' relationship is terminated.

The 'success' relationship moves to the next step of actually processing the file. For our illustration purposes we just use a 'LogAttribute' processor to simulate the processing of the file. In our case, we log the flowfile name. This completes our first part.

The wait processor needs the 'DistributedCache' services (we can set this up easily). I use a signal identifier 'testSignal'. This signal name will have to be used in the 'Notify' processor too.

We can actually run the flow now. The 'GEnerateFlowFile' will crank out files and we can see all of them actually waiting. The 'queue' from Gen to 'Wait' will be filled up.

Now, let's come to the notify part. The scenario that I am trying to simulate is like: we get a hit/response from an external source and for every hit we can process one file. To demonstrate this, I am going to use a 'HandleHttpRequest' processor. This starts a HTTP server listening for incoming requests. On getting a request, we route the 'success' to 'Notify' processor which uses the same signal text. The success of 'Notify' leads to 'HandleHttpResponse' which then completes the http request with a response code of 200.

 We can now run the entire flow. Using curl we can hit the endpoint like, http://localhost:5050 as this is a simple get request (alternatively you can paste the URL in the browser and hit refresh). If we tail the nifi-app.log file, we can see that, for every hit we do via curl/browser, the wait releases one file and the file name gets printed in the log.

The template for this example is available in my github repo.

Spring Boot Project with Angular - Development & Build

This blog post is to explain how to build a Spring Boot application with Angular as the frontend UI with Maven as build tool.

Goals:
1) During development run the server separately (spring-boot with live reload)
2) During development run the client separately (Angular with live reload)
3) During dev, Angular UI should be able to call the Sprint Boot RESTFul endpoints even when running separately
4) Build should however be a single Spring Boot jar with UI working as usual (without any change to REST calls).


We first create a multi-module Maven project named 'spring-boot-ang-sample' with pom packaging with includes two child modules, like:
<modules>
  <module>spring-boot-ang-sample-ui</module>
  <module>spring-boot-ang-sample-service</module>        
</modules>


There is nothing much else in the pom. We then create the Spring Boot project 'spring-boot-ang-sample' which is a regular spring boot project with devtools included for live reload. We also create a rest controller class with endpoint api/time/local which just returns a simple JSON with server's time, like:
{"type":"Local","serverTime":"2019-12-24T14:26:21.501"}

We can run this project with mvn spring-boot:run and when we hit localhost:8080/api/time/local we should see the response as above. Now, we are able to do the dev-test cycle for the server.

Next, we create the Angualar project 'spring-boot-ang-sample-ui'. We do this with Angular cli commands, like 'ng new'. This will create the regular angular project and we can run this with ng serve. We can then use the UI at localhost:4200. The changes we make to the UI will reflect with live reload.
The UI code makes calls to the server end points. But, we have a catch here. The server runs in a different port so, but, the calls from within Angular will go only to 4200 or the port which client runs.

Angular provides a nice feature to proxy calls to server: proxying

This is a very useful feature. So, we will make use of this to setup a proxy file wherein, all the calls for 4200/api will goto 8080. This helps us to run client and server separately in dev mode and can develop the interaction freely without worrying about code changes.

When building the project (for deployment), we will make use of the 'frontend-maven-plugin' from 'com.github.eirslett'. This plugin installs node and runs the build for the project. This is useful if you are running the build from say Jenkins where there is no node currently installed.

Remember that the 'spring-boot-ang-sample-ui' project is just an Angular project. So, let's first create the pom.xml so that it becomes a child module of the 'spring-boot-ang-sample' parent project.

Next, we change the angular output path of the angular build to go and reside in the 'spring-boot-ang-sample-service' project under 'src/main/resources/static'. This is a requirement for a spring-boot project to serve web pages. Note that we're not placing it within src/main/resources directly as the Java project resources can be kept separate and clean. The output path is changed, like:
"outputPath": "../spring-boot-ang-sample-service/src/main/resources/static",

Now, when run 'ng build' or when we run mvn clean install on the UI project, the distribution goes and sits in the static folder of the Spring-boot project. Next, when we build the Spring boot project as a jar (mvn clean install), the static resources (web pages ) become part of the jar. After the jar is build navigate to the target folder and run the jar, like:
'java -jar spring-boot-ang-sample-service-1.0.jar'

Now, we can navigate to http://localhost:8080. Note that we can't navigate to 4200 and try to access the web app. In build, the whole web application runs in 8080 and web pages are accessible.
Also, the RESTFul API (endpoints) are accessible as usual at http://localhost:8080/api.

Now, let's hit the endpoint localhost:8080/api/time/local and we should see the same JSON as we saw above. Now, navigate to localhost:8080/, you should see the web page and hitting on the button hits the server and brings back the data and displays it. Note that all along we just use Angular HTTPClient to simply access the 'api/time/local' endpoint.

If you notice above, we built the UI project first and then the server project. This is because, in the build, we need the latest web distribution to be available at src/main/resources/static before the spring boot project is built.
Hence, if you notice the parent project pom, the module order is mentioned as UI coming first and then the services. So, when we run 'mvn clean install' on the parent project 'spring-boot-ang-sample', the UI project gets built first and then the service gets built. The final jar is present in the target folder of the service.

Also note that, the api is also accessible separately should you decide to use it to access from elsewhere like mobile apps. Thus, we have achieved the 4 goals mentioned.

Important: Also, note the files that have been ignored from git commit(s). Apart from the usual 'target' folder and IDE specific files, we have also ignored the 'src/main/resources/static' folder from commit. This is because, this has to be built fresh every time to reflect the latest UI changes, so, no one should commit these files to the repo.

The complete source code is available in my Github repo

References:
https://dzone.com/articles/building-a-web-app-using-spring-boot-angular-6-and
https://dzone.com/articles/simplified-building-web-application-using-spring-b
https://dzone.com/articles/angular-7-spring-boot-application-hello-world-exam

Spring WebSocket for Specific User Without Stomp

When I started to learn how to do web sockets with Spring, I was able to get lot of examples. However, almost all of them were using STOMP protocol. STOMP is important that it gives structure to web sockets. However, I still wanted to understand how the basic web socket works before moving on to STOMP.

In particular, this post will showcase an example web socket using Spring and sending messages to targeted logged-in user(s).

Most of the code for this example has been taken from http://www.devglan.com/spring-boot/spring-websocket-integration-example-without-stomp.
We add the targeted user messaging on top of this.

We first configure the Spring web security allowing users to login to the application using form login. We use in memory authentication with sample users for demo purposes. Plus, we use the standard technique by having an endpoint to get the login user (and on 401 it redirects to login).

Run the application as a Spring boot app and go to the url: http://localhost:8080/app.html. I have configured 2 users - user1/pass1 and user2/pass2. Login using one of these and click the connect button.

Once logged in, we have a landing page where we have a button which allows us to connect to the web socket (this would be done from plain Javascript and the script is put inside app.html itself). This script is simple, uses plain Javascript to connect to web socket. And, if a message is sent by the server to the connected client, we simply receive the message and add it to the '#greetings' div which is defined on the landing.html. The message is simply appended to the this div, so, we can see all the message as it is sent by the server.

On the server side, we use the Spring @EnableWebSocket annotation and register a web socket handler for the endpoint '/' (this is not usually the case, as a practice, people would have something like, 'directws' or something to indicate this separately - like we usually use the 'rest' for rest endpoints).

The web socket handler extends the Spring TextWebSocketHandler class and overrides the afterConnectionEstablished method. We create a BroadcastService and pass on the session. The BroadcastService stores the session in a list.

The scenario we have used is where a server needs to 'push' messages to connected clients (about a new event, or about an email which has arrived and so on). So, I have added a REST endpoint to help us to simulate this. I call it PushController which has endpoints like 'push/broadcast/message' which is for broadcast and 'push/targeted/{user}/{message}' which is for specific user. So, we can deploy the server and simulate the push by making a REST call to this endpoint from say Postman or ARC. These end points in turn call the methods of broadcast service that we talked about above.

Let's go back to the broadcast service and examine the methods. We have a broadcastMessage method which just loops over all the available web socket sessions (which was created when a client connected) and calls sendMessage on those - which will in turn sends this message to call connected clients. This is quite simple and works well for broadcast.

The other method is what we are interested about. The targetedMessage method takes the message as well as the user name - to which the message is targeted. So, now how do we send the message to the specific user?

The WebSocketSession class has a method getPrincipal which returns the principal i.e the user who has logged in. This actually returns the same old java.security.Principal instance - which is really cool as it neatly ties with the Principal object that we are acustomed to in Java Enterprise Applications. So, I now loop over the available web socket sessions and if the principal's name (user name) matches with that of user name passed into the method, we just call the sendMessage on that session - which ensures that the message is pushed only to the user.

The demo is not very efficient, but, I hope it conveys the basic idea. We can expand on this to other situations - like pushing messages to users of a specific role for example.

The complete code is available in my github repo.

Crores 2 Billion

I have been wanting to write a converter that can convert not just from crores to billions (and vice-versa), but, one with conversion rates. Plus also a way to easily mention large numbers like lakh crores or crore crores.

For example, I would like to know how much in million/billion $ would be Rs.1.76 lakh crore?

And, I finally developed and deployed the following app in the Google app engine:
http://crore-2-billion.appspot.com/

Important thing is that, I have used a simpler notation for the user to enter in large amounts.
The fields on the left hand side will be in Indian Rupees and the ones on the right will be in foreign currencies. The foreign currency can be selected in the combo (rhs). The left hand side has a conversion rate (which can be adjusted by the user).

So, if you want to convert 1.76 lakh crore, you can enter 1.76 LC - LC means Lakh Crore (case insensitive). Likewise, TC for Thousand crore and CC for Crore Crore and so on. So, you want to enter our budget numbers, you can do so with a simple notation.

When the right arrow (-->) is clicked, the Indian notation is converted into million/billion. We can see two fields on the right. One which just does numbered conversion and the other which includes the rate too.

So, when you enter 1.76 LC and click the arrow, you see this converted to pure number which is 1.76 trillion and also with $ conversion - which turns out to be $27.08 billion.

Likewise, you can do the reverse. Enter 165 M (for 165 million $) and press the left arrow key (<--).
It will be converted to lakh/crore along with the rate conversion and both numbers shown.

I hope this would be useful to finance people who track big budgets.

Spring Security Authentication and Authorization with Custom Schema

In my earlier post, I described how to do authentication and authorization with PicketLink using a custom schema. In this post, I use the same schema to perform the same with Spring Security and Spring Data JPA. Spring Security provides everything we need and we can configure authentication and authorization supported from a database using queries. In this post, I show an example of how to achieve the same using Spring Data JPA repositories.

Most of these are covered in the post: http://www.baeldung.com/spring-security-authentication-with-a-database.
I build on top of this by tweaking it to use the same schema as used in my earlier post where the password is kept in a separate table:

A simple one, with a table to store the user info, a separate related table to store password (this is useful because, when bringing back user from DB, we don't need to bring back the password related info). A master table for role and another table to associate the roles for a user.

I write corresponding JPA entities for these tables. Importantly, I don't have a relation from AppUser to UserPassword entity (it's the other way around) - simply because, as mentioned above, I don't want to bring back password information when I load user. Same applies in case of roles too. One of the basic concepts to remember about security is that, information should not be provided unless and until it is asked for (strictly on a need-to-know basis).

So, how do we encrypt the password and store the same in DB? Spring provides excellent support for encoders. So, we declare a bean in the config file, such as:

@Bean
public PasswordEncoder passwordEncoder() {
  return new BCryptPasswordEncoder();
}

Let us now look at how to create an user with password and associate a role. We can create the entities and persist with Spring Data JPA repository:

String role="ADMIN";

AppUser appUser = new AppUser();

appUser.setLoginName(email);

...

appUserRepository.save(appUser);

UserRole userRole = new UserRole();

userRole.setAppUser(appUser);

userRole.setRole(roleMasterRepository.findByName(role));

userRoleRepository.save(userRole);

UserPassword userPassword = new UserPassword();

userPassword.setAppUser(appUser);

userPassword.setPasswordHash(passwordEncoder.encode(password));

userPasswordRepository.save(userPassword);

The above block of code can be placed within a transaction and we can call this service from wherever user creation needs to be done.

For the user authentication to work, we need to wire a authentication provider which can be  a DaoAuthenticationProvider as we are storing the info in a database. We need to pass an service to this provider which implements org.springframework.security.core.userdetails.UserDetailsService. 

We also need to set the password encoder which is same as we saw above. The UserDetailsService is an interface, so we need to override the method loadUserByUsername() which should return a org.springframework.security.core.userdetails.UserDetails instance. 

So, we create a class which implements UserDetails interface. I have named this as UserLoginDto and I pass the AppUser object and the role of the user. Using this the overridden methods can return corresponding values. For example, the method isEnabled() can simply return appUser.isActive().

Most importantly, we have to override the getAuthorities() method which needs to return the list of roles assigned for the user. Since we have retrieved the roles for the user from the DB, we can use that to build the Set and return it 

(note: I have used only only role per user in this example, we can easily expand on that).

public class UserLoginDto implements UserDetails {

  public UserLoginDto(AppUser appUser, String passwordHash, AppRole appRole) {

    this.appUser = appUser;

    ...

   }

  @Override

  public Collection<? extends GrantedAuthority> getAuthorities() {

    Set<RoleGrantedAuthority> roleSet = new HashSet<>();

    roleSet.add(new RoleGrantedAuthority("ROLE_"+ appRole.toString()));

    return roleSet;

  }

  @Override

  public boolean isEnabled() {

    return appUser.isActive();

  }

  ...

}

So, when Spring is going to authenticate, it will call the loadUserByUsername on the service. We use Spring Data JPA repositories of AppUser, UserPassword and UserRole to fetch these values from the DB. We then build the UserLoginDto object and return it from the method. 

Note that, we need to fetch the password at this place as it is needed to authenticate the user. However, the AppUser object is free of this password and we can pass this around back to the UI separately. This should be enough for authentication. 

For URL based authorization, we can set it up in the security config,such as:

.antMatchers("/userpages/**").hasAnyRole(AppRole.USER.toString(), AppRole.ADMIN.toString())

.antMatchers("/adminpages/**").hasRole(AppRole.ADMIN.toString())

The complete source code for this post is available in my github repo.

PicketLink Authentication and Authorization with Custom Simple Schema

I was looking for a modern CDI enabled JEE security framework and found a very good one in PicketLink(PL) as it has pretty much everything. What's more, it combines with Apache DeltaSpike for authorization which is cool.

However, I have one or two problems asociated with PicketLink's IDM structure which we use for this:

a) The ID in the IDM User class is defined as a String and not an Integer/Long. This is a problem. In most cases that we encounter, the ID is a numeric type and is referred in several other tables. Being a numeric type comes in handy particularly when writing raw queries - which is needed when debugging production issues. The String in the IDM is also a generated one and results in a long hash - which is very difficult to track and use in these types of situations.

b) Secondly, how do I use PicketLink + DeltaSpike authorization with all its goodies to an existing database - which mostly has a numeric ID field?

c) Thirdly, could I have a much simpler table structure to deal with authentication and authorization? PicketLink's IDM is robust, but, we may not need it in all cases.

There have been some successful solutions used by other people. The most common is to have a numeric ID and tie it as an attribute in the IDM scheme of things and then refer to that numeric value in other places.

In this post, I use a custom solution which I found useful and simple - which might come in handy for small application(s).

I define a typical model for user management - which is not related to the IDM schems, such as:

A simple one, with a table to store the user info, a separate related table to store password (this is useful because, when bringing back user from DB, I don't need to bring back the password related info). A master table for role and another table to associate the roles for a user.

I write corresponding JPA entities for these tables. The numeric ID field becomes the primary key and is also auto-generated via a sequence. Importantly, I don't have a relation from AppUser to UserPassword entity (it's the other way around) - simply because, as mentioned above, I don't want to bring back password information when I load user. Same applies in case of roles too. One of the basic concepts to remember about security is that, information should not be provided unless and until it is asked for (strictly on a need-to-know basis). 

Let us first create sample users and roles. I use the @Initializer class (as it is done in many PL quickstarts). First, create the records for the 'role_master' and then create users with roles. For an user, this would insert 1 record into user table, 1 record into the password table and 1 into the 'user_role' table. 

First things first, how do we encrypt the password and store the same in DB? Had we been using PL IDM schema, the PL API would have taken care of the same. 

I do almost the same as what IDM does (benefits of open source!) - make use of the org.picketlink.idm.credential.encoder.SHAPasswordEncoder class to encrypt the password (and then use JPA to store the same). This class simply uses the java.security.MessageDigest class to get the SHA implementation (so, there is no big danger of depending on a PL implementation). We pass an argument of 512 to indicate the strength of the SHA algorithm.

(sidescript: the createUser method of the @Initializer class can be used to create user when you want to add user via the UI and this method can be placed in a transaction).

Now, how do we authenticate when the user wants to log-in to the application? We have the usual login.xhtml JSF page which is same as in any PL example. For authentication, as the default PL uses IDM, we cannot rely on that. We need to write our own authenticator and wire this up with the PL ecosystem. A nice example is given at https://github.com/jboss-developer/jboss-picketlink-quickstarts/tree/master/picketlink-authentication-jsf

I follow the same and write the CustomAuthenticator class. Instead of IDM schema, I use simple JPA queries to query the user and the password tables with the user entered user id and password. As the user id is the login name in my example, I use the findByLoginName query to check against the user table (note that the numeric id that we spoke about in the beginning is obviously not used as the login id by the real users). 

As for the password, we again use org.picketlink.idm.credential.encoder.SHAPasswordEncoder instance and call the verify method after we get back the password hash from the table using JPA. 

Now comes the most important part. If the authentication is successful, we set the status as AuthenticationStatus.SUCCESS. Once we set the status as success and the page is loaded, the identity.isLoggedin would be true and the user and admin links would be displayed.

And next, we have to set the account object. This account object should be a type of org.picketlink.idm.model.basic.Account. This is in a way enforced by the API. Setting this account is both important and useful as this is the account that is retrieved via the Identity object (which can be injected). This is same as setting the user info in the session.

Now, because we are forced to use the IDM Account object (as the setAccount method takes that type), we will create an instance of org.picketlink.idm.model.basic.User class - as User extends Account. Now, we have to set the values of the User object (fn, ln, email etc.). The most important information is the ID. As outlined earlier, the ID within this IDM User object is a String. The ID that we use in our AppUser is a Long. So, we simply convert the Long to String and set it into the User's ID. 

So, now, later on, wherever we need the ID, we can simply @Inject the Identity object and get the value. For example, to get the user name, we can do:
((User)identity.getAccount()).getLoginName()

Now, to the next important part - authorization:

Authorization checks would be needed at two levels. One is at the UI side and the other at method-level checks on the server-side. Apache DeltaSpike provides a CDI enabled authorization module which blends nicely with PL. Some examples can be found in the PL quickstarts for the same.

We are going to deal with the UI side authorization checks - this again would be needed at two levels. First is at the displayed UI, where a part of the UI is hidden/disabled for certain roles. Second, at the URL level.

Within the UI, there are a few types of authorization needs. One is that showing/hiding or enabling/disabling parts of the UI for specific roles. Let's see how this works. This is actually very simple. Login as 'user1' and goto the common page - you should see the 'Save' button in disabled state. Now, login as 'admin1' and goto the common page. Now, you should see the 'Save' button enabled. 

This is achieved simply using JSF EL. The JSF EL uses a named bean 'authChecker' (AuthorizationChecker.java) on the isAdmin method. The isAdmin method in turn queries the 'UserRole' table using a JPA query to check if the user has the specified role. 

This is cool. Likewise, you can navigate to the admin page. The link to the admin page (menu) shown on the home page can also be shown/hidden based on the role. But, what if the user copies the admin url and pastes it into the address bar after being logged in as a normal user 'user1'? So, we need URL level authorization to happen here.

PL does supports URL level authorization in a simple manner. However, there are some issues here. Since our whole model is customized and not based on IDM, the custom authorization of URLs for a specific role does not work. For example, the following does not work:

When we build the security configuration, we can make use of the forPath() and authorizeWith() methods to specify URL authorizations. For example, we use the following in our code:

  .forPath("/faces/admin/*")

  .authorizeWith().role(AppRole.ADMIN.toString())

But, how and where do we hook the role into PL? Remember that in the custom authenticator that we wrote, we set the status and the account. But, nowhere did we set the roles for the logged in user. And, there is no method on the PL Account class to set the roles for the user.

This is usually the norm. Roles are not set anywhere. Security works on a need-to-know and deny-first principles and so the hasRole and such methods are provided. Still, we need to figure out on how this will work.

We can write a custom authorizer for URL level authorization too as follows:
.forPath("/faces/admin/*")
.authorizeWith().role(AppRole.ADMIN.toString()).authorizer(CustomPathAuthorizer.class)

This class needs to override the authorize method and return true/false. So, we can write our custom auth checks inside this method and return the value accordingly.This method takes a PathConfiguration as a parameter and we can get the roles with the following call:

pc.getAuthorizationConfiguration().getAllowedRoles();

And then with that, we can do our custom auth check. More info can be found from org.picketlink.authorization.DefaultAuthorizationManager source.

However, when I implement this and run, no matter what role I am logged on to, going to the admin URL directly simply fails and I get a 401/403. I began wondering if this was a bug and posted in the forum (https://developer.jboss.org/thread/272838)

Let's take a step back and see how the default PL works first. There are 4 authorizers built by default and added which are called one by one to do the check (authorize method which returns a boolean). If any of them return false, authorization will be denied.

When we add the custom authorizer, even though it is added as a 5th one, it seems, one of the default 4 returns false and so we continue to get access denied. And all the default authorizers are tied to the IDM schema.

I found a workaround thanks to a tip in one of the forum messages. The tip was to not use the 'role()' method at all. So, I tried that and the authorizer is working fine with the same now. Just use:
.authorizeWith().authorizer(CustomPathAuthorizer.class)

So, where do I specify the roles for the URL? Inside the authorizer, I can get the pathInfo from the request and then use it to check with my roles. For example, in the authorize method, I can get the path and then check against a List/Map to verify if this path is allowed for the role of the logged in user. As an example, I have used a trivial sample map with path and list of roles. For more robust ones, we can propably have this data also in the database.

In the current sample, I just loop against the paths to match and then see if the logged-in user has the requisite role to go to the URL. To test this, login as a user and then paste the admin URL directly in the browser. You should see an access denied page.

The complete sample code is available in my github repo.

JavaFX TreeTableView Example with Different Entities

In an earlier post, I had showed how to build a tree-table component (in Swing with SwingX JXTreeTable component) with 2 different entities (which have a HAS-A relationship). In this post, let us examine how to achieve the same in JavaFX TreeTableView.

As indicated in my earlier post, in real scenario(s), we will need to deal with showing a tree-table view with different entities, for example, to display a department with list of employees belonging to the department. These may share a relationship at the db level, but it will be a HAS-A relationship at object level. When we attemp to display this in a tree-table, note that the parent is one type of object and the children (leaf) another.

The official JavaFX TreeTableView tutorial available on Oracle site, actually does display a department and a list of employees as children. But, if you notice the code, it uses only one object - an Employee object throughout (even for the department). The department is just displayed as a trick of using the first object as root (where the name is set and email is left as empty). This is good for illustration purposes, but, it is important to know how to deal with 2 different objects for such scneario(s).

Let us first create the entities. I will be using the same attributes (except photo for Employee) from the earlier example.

public class Employee {

    private int id;
    private String name;
    private Date doj;

    public Employee(int id, String name, Date doj) {
        this.id = id;
        ...
    }
    //setters and getters not shown for brevity
}

public class Department {

    private int id;
    private String name;
    private List<Employee> employeeList;

    public Department(int id, String name, List<Employee> empList) {
        this.id = id;      
        ...
    }

    public List<Employee> getEmployeeList() {
        return employeeList;
    }

    public void setEmployeeList(List<Employee> employeeList) {
        this.employeeList = employeeList;
    }

    //other setters and getters
}

As JavaFX came into existence after Java 1.5 (the Java API version), everything in JavaFX is generified. So, right from declaring the TreeTableView to adding columns, everywhere the entity is specified. However, for us, we will be dealing with 2 different entities. So, we will not specify the entity anywhere. We will just use the normal syntax for dealing with objects like:
TreeTableView treeTableView = new TreeTableView(dummyRoot);
instead of
TreeTableView<Employee> treeTableView = new TreeTableView<>(dummyRoot);

As we are going to show a list of departments (where each department will have a list of employees), I am using a dummy root object (this root object can represent an Organization when we want to show the root), like:
final TreeItem dummyRoot = new TreeItem();

 Now, I create the TreeTableView with this dummyRoot tree item. Then I call the setShowRoot(false) to hide the root. Again, note the lack of usage of generics here.

Now, we need to add all our items to this root. We will deal with that later.

Let's now create the columns. We need to show 3 columns (id, name and doj) for Employee. For Department, the 3rd column will not display any value. First, I create the column, like:
TreeTableColumn idColumn = new TreeTableColumn("Id");

Next step is to specify how the column is going to fetch the value from the entity and display it.
We need to set the cellValueFactory by passing a javafx.util.Callback which will be type parameterized as follows:
Callback<TreeTableColumn.CellDataFeatures<S,T>, ObservableValue<T>> 

The Callback interface has one method call() which we need to override. The call() method actually takes TreeTableColumn.CellDataFeatures<S,T> as param and returns ObservableValue<T>. Here, S is the parent object that represents that row and T is the type for that column.  This will be called back whenever JavaFX decides to update its view.

As is the norm in JavaFX, we would normally generify the call with the object that we expect, say Employee and the column type, say, String. However, in our case, as the parent object can either be Employee or Department, we will not be able to use generics. Likewise, the column may actually have Integer type for Employee object and String type for Department object (not in this example, but, this is very much possible).

So, we will simply use the Object as the type for the parameter to the call method, So, the implementation will be like:
idColumn.setCellValueFactory(new Callback() {
    @Override
    public Object call(Object obj) {
            //return ObservableValue (without type)
}
});

Remember that as mentioned above, the incoming Object obj is actually of type TreeTableColumn.CellDataFeatures. So, we have to type cast the same and then call getValue() which will then return a TreeItem object:
((TreeTableColumn.CellDataFeatures)obj).getValue()

TreeItem represents each node in the Tree and has a value attached to it - which is actually the value object. So, we have call getValue() on the TreeItem object to get the domain object which in our case can be Employee or Department:
Object dataObj = ((TreeTableColumn.CellDataFeatures)obj).getValue().getValue();

So, now we can do an instanceof check on dataObj to identify the correct type and then invoke the correct method, like:
((Department)dataObj).getId()

We need to return a ObservableValue from the method, so, we will return a ReadOnlyStringWrapper which takes a String value as an argument. So, we can return like:
return new ReadOnlyStringWrapper(String.valueOf(((Department)dataObj).getId()));

The full call looks like:
idColumn.setCellValueFactory(new Callback() {
@Override
public Object call(Object obj) {
Object dataObj = ((TreeTableColumn.CellDataFeatures)obj).getValue().getValue();
if(dataObj instanceof Department) {
return new ReadOnlyStringWrapper(String.valueOf(((Department)dataObj).getId()));
}
else if(dataObj instanceof Employee) {
return new ReadOnlyStringWrapper(String.valueOf(((Employee)dataObj).getId()));
}
return null;
}
});

We can use lambda expressions for the same, which I have done for the dojColumn:

dojColumn.setCellValueFactory((Object obj) -> {

final Object dataObj = ((TreeTableColumn.CellDataFeatures)obj).getValue().getValue();

if(dataObj instanceof Employee) {

return new ReadOnlyStringWrapper(((Employee)dataObj).getDoj().toString());

}

return null;

});

Remember that, there is no doj value present for Department. So, we do an instanceof check only for Employee and return the employee's doj. Otherwise, we simply return null. So, for the rows where the Department object is displayed, the doj column will simply be empty.

Once all the columns have been defined, we can add this to the tree as follows:
treeTableView.getColumns().setAll(idColumn, nameColumn, dojColumn);

As we created the TreeTableView with a dummyRoot TreeItem object, all we need to show data is to simply add TreeItem objects as children to this root and in a recursive/tree manner to add children of children. Remember that the Department object contains a List<Employee> within itself. We create two such Department objects and add them to another list called deptList. Adding data to tree is accomplished as follows:
//add data to tree
List<Department> deptList = buildData();
deptList.stream().forEach((department) -> {
final TreeItem deptTreeItem = new TreeItem(department);
dummyRoot.getChildren().add(deptTreeItem);
department.getEmployeeList().stream().forEach((employee) -> {
deptTreeItem.getChildren().add(new TreeItem(employee));
});
});

When I run the application, it looks like this:

When comparing with my earlier post on achieving the same with Swing, I have to say that doing the same is much more simpler in JavaFX as I deal with individual columns one at a time. And, I have to implement only one method that is equivalent of getValueAt. I don't have to deal with the getChild() and getChildCount() methods as in Swing.

The complete source code for this sample can be found in my github repo.

I18N Tutor

Several years back I developed a small desktop application in Swing which helps the user (who is a Java developer himself) to understand Internationalization in Java. I called it a 'Practical' tutor. Unlike usual text based tutorials, being a Java application itself, this allows the developer to read as well as try out the examples side-by-side. I passed it on to many of my friends/colleagues and they all liked it.

The formatting was explained for Number, Decimal formats with pattern, Currency format and percentage. Along with providing input numbers, the developer could also change the locale and see how that changes the output of the formatters.

Today, I am happy to have ported it to a web page running on Google App Engine. Do try it out and pass on the link if you like the same:
https://i18n-tutor.appspot.com

NetBeans Diff API Extracted as a Standalone Project

Inspired by Emilian Bold who extracted the Progress bar API as a separate OSS project out of NetBeans source code, I started to work with the aim of extracting the Diff API out as a separate project. I had this thought in 2012 or so and it has taken 3 years to actually make it (and only the textual part!)

In the meantime, came to know that the amazing Geertjan had done something better at https://github.com/GeertjanWielenga/netbeans-visual-diff-standalone
(which I got to know via a post at http://forums.netbeans.org/viewtopic.php?t=63551&highlight=).

Nevertheless, I went ahead and completed the task that I started and also posted the code in my github repo. You can take a look if you are interested. Also, I created a test project at github repo which you can follow to make use of this library.

Also, would like to note down my experiences when trying to make a NetBeans project into an OSS which might be useful for others:

My idea was to make the diff library work with jut JDK. So, wherever I encountered external dependencies, I modified them, such as:

a) NbBundle replaced with ResourceBundle
NbBundle is used quite a lot. I replaced it with ResourceBundle, like:
ex: return ResourceBundle.getBundle("Bundle").getString("BuiltInDiffProvider.shortDescription");

b) org.openide.ErrorManager

This is used in logging exceptions. The Javadoc provides a general guidline where  clearly provides the simple alternative - "Rather then using the {@link ErrorManager} consider using JDK's {@link Logger}

 * for reporting log events, unwanted exceptions, etc."

So, I followed the advice and did the same.

c) ImageUtilities.loadImage:

This was from the package org.openide.util.ImageUtilities - source shows that class loader and ImageIO are used to load images plus provides caching.

I have simply used classloader with ImageIO.

d) The API uses NetBeans lookup mechanism throughout even though both interfaces and implementations are present in the same diff module. There are however, two ways of using lookup to find services - one is the usage of "@org.openide.util.lookup.ServiceProvider" annotation to mark the implementation. This is pretty straightforward and maps to java.util.ServiceProvider usage.

The other way used is also via using Lookup, but, registration of services is done in an old manner (which I think is deprecated now). This is explained in http://wiki.netbeans.org/DevFaqLookupDefault and http://wiki.netbeans.org/DevFaqSystemFilesystem. 

How did I find this? After reading the above links, I happened to look at the manifest.mf file - which has a reference to "OpenIDE-Module-Layer: org/netbeans/modules/diff/mf-layer.xml"

So, I opened the mf-layer.xml where the DiffProvider implementations are provided and also refer to other .settings file(s). These files in turn have the implementation class name.

It took quite a bit of time to figure this out.

Initially, I started with trying to have the complete visual part as a separate project, but, then decided to do only the textual part. For this, some of the useful code was present as part of the visual stuff. So, while working on this, I also extracted out some inner classes into top-level classes.

For example, the TextDiffVisualizer class has utility method 'differenceToLineDiffText' - which is very useful. I moved this out to a separate class named Util.

Finally, completed this and posted the code at my github repo

If I start a software company

If I start a software company...
I actually want to rant against some stuff being done in software companies...but instead of just sounding negative, I thought about what I would do if I run my own software company.
I list my thoughts about the same - first the rant and then on what I will do...

System:
I have always wondered about this. Executives would be given high-powered brand new Mac, but developers would be given a normal system with 2 GB RAM and a ok processor. Most of the work of executive is in his/her head - system is used mainly for mail access, documents, sheets and ppts. Isnt it?

I never really understood this. Particularly, being from a Java background, typically such systems require higher RAM. Usually, we will run the IDE, and also run the app server in our system along with a database too. I remember when working for a MNC, we got a system with 1 GB RAM and it would take ages to deploy. Request to upgrade to 2 GB RAM would go through a long bureaucratic route with the PM explaining the costs involved.

Isn't a good computer the most important tool for a developer? After the people themselves, it is the system where the most investment should go. And, if the people are the most important asset in software industry (as most companies claim), should we not give them the best tools to work with? Howmuchever you provide a fancy working place with good air conditioning, a good system is the most important thing for a developer - and should I mention that a slow unresponsive system is the most annoying thing for a developer.

So, my first priority would be to give the best system to my developers.

Email, attachments, sharing of files:
These are actually inter-related.

Email:
Yes, the most trivial one. But, the most important one too. In these days of everything-on-web, I just don't understand why companies spend to buy licenses for desktop mail clients. Why can't they just use OSS mail clients in that case? Wouldn't that save a lot of cost?
Or why not simply go for a solution like gmail for business? you just need a browser for that.

Attachments & File Sharing:
I have done this several times. Let's say we are working on an estimate and prepare a sheet. I will send this to my peer attaching the sheet. They will make changes, rename the file 'v2' and send back to me. I will again rename the file like file_ranga_v2.xls and take another copy. Very soon, there will be numerous copies of the file in my hard disk (not to mention the versions circulating in emails). If I were to revisit this a month later, I would struggle to find out which is the latest version. I will have to look at the modified date and find out. Ditto for docs, ppts. Also, imagine the size of these attachments circulating around filling up mail boxes. It soon becomes a pain.

I worked with Google sheets for some time with a client and I was amazed at how easy it was to share sheets. No hassles, no multiple versions and no sending around attachments wasting space and network bandwidth. On top of it, everything is on cloud and I could access it from my personal system or smart phone too. And, it makes our day-to-day operations easy to manage. Pretty simple, but, I still see many companies (even new ones) opting for say MS-Exchange and then buying license for MS-Outlook. Pray, why?
(Not to mention the license cost for MS-Office - I forgot about that)

Thinking about this, it also helps IS (Information Security) in a way - no data needs to be stored in individual machines.

Issue Tracking via Excel Sheets:
This is similar to the above one. This is actually ironical. Many companies use excel sheets to manage issue tracking, timesheets etc. I thought excel sheets were used 10 years back. Irony is that, we do write software for many customers where requirements go like: 'our folks are currently using excel sheets. we want to replace this with a system'!!
(I think the customers do not know that these companies actually use a excel sheet and not a system - if they come to know about it, they will actually think twice about awarding such projects!)

And imagine doing this in the above mentioned fashion - numerous versions circulating around in mail boxes. And, how do we derive metrics out of these? We will have to first identify the correct version and then gather data. Systems, on the other hand, generate reports with a click of a button. And are searchable too. And, you don't have to re-invent the wheel. Numerous OSS projects are available for such purposes in most technologies. In fact, we are spoilt for choices.

At the least, use something like Google sheets!!

Software and anti-OSS:
Some companies have a policy of no Open Source Software. Yes, it's true. However, you could see such companies using a lot of 'trail' software. I have seen many places where 'EditPlus' (a popular editor) being used - without license being bought. This particular software allows you to use it, but with an annoying warning window when it starts (which clearly states that you can use it only for 30 days). I don't get this. If it is so important for you, why don't you just buy a license. It is not just illegal to use software without buying a license. It is also unethical, unfair & disrespectful in my opinion - what these people are basically saying is: we earn money by writing software for our customer, but, we will not respect the other company/person who has spent lots of time developing this software. If you feel the cost is prohibitive and you won't make any margin, then, just don't take up such projects where there is dependency on high-cost licensed software.
Or, just use OSS and drop all pretensions about the same.

In my own company, I will make maximum use of OSS right from the Operating System. I have myself been using Ubuntu in my home machine for a few years and it is perfect for development - much much better than Windows - I don't have to restart my system after an install not to speak of the breeze to install stuff with apt-get. And, till date I have never seen a command equivalent to grep/egrep/find with  its unmatched speed - try searching for a file in Windows.
This covers all technologies except MicroSoft. Companies working on projects with MS technology should buy appropriate license(s).

Tools/IDE:
I am a tools guy. I believe developers can be more productive using tools. I have seen many companies impose tools/IDE on developers - they call it a 'standard'. And, many companies also buy expensive software/IDE which is such a waste - when the same could be done with OSS tools.

If the build system is independent, why can't we allow the developers to use their favorite tools? And, the sheer number of tools available is very high.

In my own company, I will strive to choose the build system for the project which is based on command-line - this would allow the developers to use their favorite tool to write code. In fact, this is a must. This allows us to introduce CI which helps in achieving higher quality and follow agile.

Laptop/Desktop:
In an earlier life where I worked at an MNC, I was given a desktop (yah, 1 GB RAM). And then, during conference calls, we would go over to a meeting room. I would need frequent access to my system to answer questions efficiently, but obviously, I can't carry my desktop to the meeting room. It used to be a pain. Some companies solve this problem by having a desktop in meeting rooms and doing a RDC to the developer's system.
I would rather provide laptops to all my developers (and not just managers, leads). The cost difference is not much these days. Only problem with laptops is that they have to be guarded against theft.

VOIP calls:
Of course, MNCs have the costly VOIP phones. I would prefer to use GTalk/Skype. I would also like my developer to take calls from home if need be - late evening or early morning calls. It is actually much more productive and helps in work-life balance.


I have tried to cover as much points in a random fashion. Of course, I have not said anything about getting projects. That is because I do not know anything about it. You might of course question me like, 'dude, you need projects to run a software company'. True. I have no idea how to do that. But, I do believe there are online avenues for these. Of course, it won't be easy.

The reason I have talked only about operational aspects is simply because - these are the ones that affect me on a day-to-day basis. And, I do strongly believe that, operational efficiency can make a major difference to an organization. You could ask about what is the USP of my software company. True, not much. But, operational efficiency can actually be the difference between success and failure of project(s). At least, that is what I think.