Pages

Saturday, June 29, 2013

Night Trek at Khajaguda (!Supermoon)

Yes it was supermoon night but cloud cover ruined it it. While it is proved that supermoon is one of the most over credited astronomical event , still its one of the reason to go out and enjoy the nature at night. Thanks to city light, tough there was no moonlight , we could see the path and do the trek without a torch. We climbed some rocks as well (scrambling). It was monsoon and great time to trek.Thanks to organizers of hats club for holding this meetup and Rajesh for the photos from his new Sony DSLR.

DSC07083

Wednesday, June 26, 2013

Google AppEngine Channel API

I am not a big fan of page refresh for getting new data from server. AJAX has been the go to technology for these kind of requirements. However its unidirectional. For two way communication where server push is required , we have seen many technologies like comet etc. Websocket and WebRTC has been really cool technologies which helps sending data from server and client real time. These needs special server code for handling such requests. I have already worked on JSR 356 for websocket and glassfish reference implementation (tyrus) earlier. However in Google Appengine provides channel APIs for bidirectional communication. While client to server communication is still over HTTP GET or POST, sever creates a specific "channel" and enables itself to push data any time to specific clients. Under the hood , its actually the client which keeps polling with GET requests for new data to be sent by the server. In any case this API can be use fill in real time game servers.

The server side code

I added the following code to the example given in the previous post to use channel APIs

[code language="java"]
private void boradCastNotes(Request request,Note note) {
ServletContext context = hsr.getSession().getServletContext();
HashMap<String,ChannelPresence> liveUsers = (HashMap<String,ChannelPresence>)context.getAttribute("liveUsers");
if(liveUsers != null){
ChannelService channelService = ChannelServiceFactory.getChannelService();
ObjectMapper mapper = new ObjectMapper();
System.out.println("List of connected client ... ");
String noteStr = null;
try {
noteStr = mapper.writeValueAsString(note);
} catch (IOException e) {
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
for(ChannelPresence cp : liveUsers.values()){
System.out.println(cp.clientId());
System.out.print(" Sending message to client --> " +noteStr);
String channelMessageStr="{"command":"note","data":"+noteStr+"}";
System.out.print(" Sending message to client after wrapping --> " +channelMessageStr);
channelService.sendMessage(new ChannelMessage(cp.clientId(),channelMessageStr));

}
}
}
[/code]

Client Code

In HTML I added an extra button which will send the user entered data using an AJAX request and some JavaScript code to create the channel using the token issues by the server earlier.

[code language="JavaScript"]
<script language="JavaScript">

//Function called when update button us pressed
//This will maken an AJAX POST request
//To the webservice created using sitebricks
postnote = function(){
var noteObj = new Object();
noteObj.text=document.forms[0]['note.text'].value;
sendMessage('/notes','POST',JSON.stringify(noteObj));
};
//Generic method for sending any Ajax request
sendMessage = function(path, method,param) {
var xhr = new XMLHttpRequest();
xhr.open(method, path, true);
//Callback when response is received from the server
xhr.onload = function () {
console.log(this.responseText);
document.forms[0]['note.text'].value="";
};
xhr.send(param);
};

onOpened = function() {
sendMessage('/notes','GET','command=open');
};

onMessage = function(message){
//When message is recived from the server
//Message.data contains the actual string send by the
//Java code
//Now convert the JSON string to a Javascript Object
var data = eval("(" + message.data + ")");
console.log("Received data from Server "+data)
//Adds a new note row in the tables
if(data.command=="note"){
insertRow(data.data)
}
}

insertRow = function(data){
var table=document.getElementById("noteTable");
var row=table.insertRow(1);
var cell1=row.insertCell(0);
var cell2=row.insertCell(1);
cell1.innerHTML=new Date(data.date);
cell2.innerHTML=data.text;
}

//Opens a channel with server with the given token (provided by the server)
//Internally it keeps polling the server for new messages
channel = new goog.appengine.Channel('${token}');
socket = channel.open();
//Define all the listeners
socket.onopen = onOpened;
socket.onmessage = onMessage;
socket.onerror = onError;
socket.onclose = onClose;
</script>
[/code]

Other features

To track the clients which gets connected to the server , listeners can be added to a specific urls.I track the clients to broadcast the messages to all the connected clients.To enable tracking following needs to be added to appengine-web.xml

[code language="XML"]
<inbound-services>
<service>channel_presence</service>
</inbound-services>
[/code]

Then write POST endpoint handlers

[code language="Java"]
public class TrackerServlet extends HttpServlet {
@Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
ChannelService channelService = ChannelServiceFactory.getChannelService();
ChannelPresence presence = channelService.parsePresence(req);
System.out.print("Client trying to connect with ID " + presence.clientId());
//Save the new client in servlet context
ServletContext context = getServletContext();
//Object liveUsers = context.getAttribute("liveUsers");
HashMap<String, ChannelPresence> liveUsers = (HashMap<String, ChannelPresence>) context.getAttribute("liveUsers");
if (null == liveUsers) {
System.out.println("Initialising client list");
liveUsers = new HashMap<String, ChannelPresence>();
context.setAttribute("liveUsers", liveUsers);
}
if(liveUsers.containsKey(presence.clientId())) {
System.out.println("Err.... this guy was already connected ! ");
} else {
liveUsers.put(presence.clientId(), presence);
System.out.println(" New client connected with ID " + presence.clientId());
}
}
[/code]


Similarly remove the client from servlet context when client gets disconnected.

[code language="Java"]
public class TrackerServlet1 extends HttpServlet {
@Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
ChannelService channelService = ChannelServiceFactory.getChannelService();
ChannelPresence presence = channelService.parsePresence(req);
System.out.print("Client disconnected with ID " + presence.clientId());

ServletContext context = getServletContext();
HashMap<String, ChannelPresence> liveUsers = (HashMap<String, ChannelPresence>) context.getAttribute("liveUsers");
if (null != liveUsers) {
if (liveUsers.containsKey(presence.clientId())) {
liveUsers.remove(presence.clientId());
System.out.println("Client was disconnected");
} else {
System.out.println("Client was not connected");
}
} else {
System.out.println("No client was ever connected");
}
}
}
[/code]


Note:

 

Thursday, June 20, 2013

Appengine.Maven.Guice.Sitebricks.Objectify

I have been trying to read and use all these technology for my project but I did not find any single article/blog which includes setup instructions for all of them.

Most of these technologies are from Google and optimizes for their PaaS solution.

  • Google AppEngine - The platform as a service supporting several languages including  Python, Java.

  • Maven  - Build tool like ant but lets you pull the libraries from the original repository dynamically at the build time.

  • Guice - Dependency Injection tool like spring without XML configuration. Configuration is done in code itself.

  • Sitebricks -  Libraries for creating REST webservices and dynamic HTML page (separating HTML and Data ).

  • Objectify - Library to interact with Google AppEngine datastore and automatic memcached management.


Following are the major steps I took for creating a working project.

  1. Since I was trying to create a web app with maven I needed to create the folder structure for web project containing WEB-INF etc. I could do this manually too. Details .
    $ mvn archetype:generate -DgroupId=com.neil -DartifactId=NoteWebApp -DarchetypeArtifactId=maven-archetype-webapp -DinteractiveMode=false


  2. Added the Google appengine dependency in the POM.xml in the recently created Maven project.
    [code language="xml"]
    <dependency>
    <groupId>com.google.appengine</groupId>
    <artifactId>appengine-api-1.0-sdk</artifactId>
    <version>1.8.1</version>
    </dependency>
    [/code]

  3. Add Google app engine plugin for the maven tools to be used for running devserver and uploading the build to appengine cloud.
    Here we could also mention the debug port which any idea can connect.

    • Local server can be started using "mvn appengine:devserver". Details.

    • Local app can be deployed in appengine server at "mvn appengine:update"


    Sometimes "port in use" error occurs if previous debug port or server port is not closed gracefully then we have to query for the process using the port and kill it.
    sudo lsof -i :8080 # checks port 8080 in mac
    kill -9 2828

    [code language="xml"]
    <plugins>
    <plugin>
    <groupId>com.google.appengine</groupId>
    <artifactId>appengine-maven-plugin</artifactId>
    <version>1.8.1</version>
    <configuration>
    <!--
    <jvmFlags>
    <jvmFlag>-Xdebug</jvmFlag>
    <jvmFlag>-agentlib:jdwp=transport=dt_socket,address=5000,server=y,suspend=n</jvmFlag>
    </jvmFlags>
    -->
    </configuration>
    </plugin>
    </plugins>

    [/code]

  4. Add the appengine-web.xml in the same directory as web.xml which will hold the appengine configuration.
    [code language="xml"]
    <appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
    <!-- create this unique id in appengine console in web-->
    <application>testjavaneil</application>
    <!-- I only keep only one version during dev and keep overwriting it. -->
    <version>1</version>
    <!-- I have no idea about it -->
    <threadsafe>true</threadsafe>
    </appengine-web-app>
    [/code]

  5. Add dependency for Guice
    [code language="xml"]
    <dependency>
    <groupId>com.google.inject</groupId>
    <artifactId>guice</artifactId>
    <version>3.0</version>
    </dependency>
    [/code]

  6. Add a listener servlet in web.xml which will get executed when container comes up. Also add a filter which will redirect all the urls to the Guice filter
    which in turn will have the information which class to be executed depending on the request URL.This configuration will be done in the listener.
    [code language="xml"]
    <filter>
    <filter-name>webFilter</filter-name>
    <filter-class>com.google.inject.servlet.GuiceFilter</filter-class>
    </filter>

    <filter-mapping>
    <filter-name>webFilter</filter-name>
    <url-pattern>/*</url-pattern>
    </filter-mapping>

    <listener>
    <listener-class>com.neil.MyGuiceServletConfig</listener-class>
    </listener>
    [/code]

  7. As promised in the last step I will create the listener with the information of url mappings
    [code language="java"]
    public class MyGuiceServletConfig extends GuiceServletContextListener {

    @Override
    protected Injector getInjector() {
    return Guice.createInjector(
    //Keep sending Guice the modules
    new SitebricksModule() {
    @Override
    protected void configureSitebricks() {
    scan(NotebookService.class.getPackage());
    //Should change this to logger, this is just to proove that
    //sitebrick scans the classes for annotations like @At etc.
    System.out.println("****** Scan complete ******");
    }
    }
    , new ServletModule() {
    @Override
    protected void configureServlets() {
    //Servlet classes have to be singleton to be consistent with servlet specification
    //In tranditional cases web.xml config tell the container to do so I guess.
    bind(com.neil.NotebookServlet.class).in(Singleton.class);
    //Analogous to typcial servlet URL mappings
    serve("/servlet").with(com.neil.NotebookServlet.class);
    }
    }
    );
    }
    }

    [/code]

  8. Following is a typical servlet class but will will avoid this and use templates and webservices to render data in HTML or JSON format.
    [code language="java"]

    public class NotebookServlet extends HttpServlet {

    //Register the entity class for data persistance service
    static {
    ObjectifyService.register(Note.class);
    }

    /**
    * Get requests come here
    *
    * @param req
    * @param resp
    * @throws ServletException
    * @throws IOException
    */
    @Override
    public void doGet(
    HttpServletRequest req, HttpServletResponse resp)
    throws ServletException, IOException {
    resp.setContentType("text/html");
    //Render the form for adding notes
    resp.getWriter().println(
    "<form method=post action="/servlet" >" +
    "<input name="note.text" size="20" type=text/>" +
    "<input type=submit value="Add Note">" +
    "</form>");

    resp.getWriter().println("List of notes");

    //load all the data from datastore
    List<Note> notes = ObjectifyService.ofy().load().type(Note.class).list();
    //Render the notes in each row of the table.
    resp.getWriter().println("<table><tr style="background:grey"><th>Date</th><th>Note</th></tr>");
    for (Note noteEntry : notes) {
    resp.getWriter().println("<tr>");
    resp.getWriter().println("<td>" + noteEntry.getDate().toString() + "</td><td>" + noteEntry.getText() + "</td>");
    resp.getWriter().println("</tr>");
    }
    resp.getWriter().println("<table>");
    }

    /**
    * Handles the form submit post requests and redirect to the same get request to display the list of
    * notes
    * @param req
    * @param resp
    * @throws ServletException
    * @throws IOException
    */
    @Override
    public void doPost(HttpServletRequest req, HttpServletResponse resp)
    throws ServletException, IOException {
    Note note = new Note();
    note.setDate(new Date());
    note.setText(req.getParameter("note.text"));
    ObjectifyService.ofy().save().entities(note).now();
    doGet(req, resp);

    }

    }
    [/code]

  9. Create the entity to map the database
    [code language="java"]
    @Entity
    public class Note {
    @Id
    private Long id;
    private Date date;

    public Long getId() {
    return id;
    }

    public void setId(Long id) {
    this.id = id;
    }

    private String text;

    //rest of the getters and setters
    }
    [/code]

  10. Create the class with sitebrick and annotate as service. annotate the url mapping and get post methods
    [code language="java"]
    @At("/notes")
    @Service
    public class NotebookService {

    private Note note = new Note();
    //Register the entity class for the Objectify persistance service.
    public NotebookService() {
    ObjectifyService.register(Note.class);
    }

    @Get
    Reply<List<Note>> showNotes() {
    //Prepare the HTTP headers
    Map<String, String> headers = new HashMap<String, String>();
    headers.put("Content-Type", "application/json");
    //Fetch data from database
    List<Note> notes = ObjectifyService.ofy().load().type(Note.class).list();
    //Convert the entity object to JSON and return.
    return Reply.with(notes).as(Json.class).headers(headers);
    }

    public Note getNote() {
    return note;
    }

    public void setNote(Note note) {
    this.note = note;
    }

    /**
    * Post request endpoint here inserts data in database
    * and returns the JSON of the single entry which was newly
    * created
    *
    * @param request the body of the request containing data is
    * obtained from here.
    * @return
    */
    @Post
    public Reply postNote(Request request) {
    Map<String, String> headers = new HashMap<String, String>();
    headers.put("Content-Type", "application/json");
    //Read JSON data and create entity object
    note = request.read(Note.class).as(Json.class);
    //System generated date instead of user
    note.setDate(new Date());
    //Store data
    ObjectifyService.ofy().save().entities(note).now();
    //Just return the newly added data
    return Reply.with(note).as(Json.class).headers(headers);
    /*
    //to redirect to a url but this class is only for
    //webservice call, we don't ahve to redirect to any page
    return Reply.saying().redirect("/servlet");
    */
    }
    }
    [/code]

  11. For using sitebricks with HTML template create another class and HTML
    [code language="java"]
    @At("/webnotes")
    @Show("/notes.html")
    public class Webnote {

    //When HTML page is rendered , this instance variable would be used for the placeholders to populate
    private List<Note> notes;
    //Following instance variable will be populated when form is submitted from the HTML template
    //The name of the input fields will be mapped to the entity components
    private Note note = new Note();

    @Get
    public void get() {
    this.notes = ObjectifyService.ofy().load().type(Note.class).list(); //load from db
    }

    public Note getNote() {
    return note;
    }

    public void setNote(Note note) {
    this.note = note;
    }

    @Post
    public String post() {
    //Date is not provided by the form, server date is populayted
    note.setDate(new Date());
    ObjectifyService.ofy().save().entities(note).now();
    //Redirect to same class and render the same HTML template
    return "webnotes";
    }

    public List<Note> getNotes() {
    return notes;
    }

    public void setNotes(List<Note> notes) {
    this.notes = notes;
    }
    }

    [/code]
    [code language="java"]
    <!--
    HTML Template for sitebricks. This has the html form and table to enter and display the data
    However theer are placeholders for sitebricks to replace the data before serving to client
    -->
    <html>
    <head>
    <title></title>
    </head>
    <body>
    <form method=post action="/webnotes">
    <input name=note.text size=20 type=text/>
    <input type=submit value="Add Note">
    </form>
    <br>

    <table>
    <tr style="background:grey">
    <th>Date</th>
    <th>Note</th>
    </tr>
    @Repeat(items=notes, var="note")
    <tr>
    <td>${note.date}</td>
    <td>${note.text}</td>
    </tr>
    </table>
    </body>
    </html>
    [/code]

  12. Add the objectify dependencies in POM.xml
    [code language="xml"]
    <dependency>
    <groupId>com.googlecode.objectify</groupId>
    <artifactId>objectify</artifactId>
    <version>4.0b3</version>
    </dependency>
    [/code]

  13. Register the objectify Entity class as service whenever we create any service which will have database interaction. I do it in constructor of the service class
    [code language="java"]
    public NotebookService() {
    ObjectifyService.register(Note.class);
    }
    </li>
    [/code]

  14. Read/write data using Objectify. Note.class is the entity class
    [code language="java"]
    List<Note> notes = ObjectifyService.ofy().load().type(Note.class).list();
    [/code]


    [code language="java"]
    ObjectifyService.ofy().save().entities(note).now();
    [/code]


I have put all the code in Github and its still evolving, however you can get hold of the particular commit.