Monday, October 27, 2014

Thursday, October 9, 2014

My name sent to mars



When I saw last 240,543 forms had been submitted for boarding passes. Get yours here. Last Day to Register: Oct. 31, 2014

Tuesday, August 26, 2014

Load Testing Using Gatling

Gatling is an open source load testing framework based on Scala programming language. One of the advantage of Gatling over other load testing framework that I have used is that the code is very elegant and readable, thanks to its own DSL. One may not have prior knowledge of Scala to get started with Gatling. There are some documentation in the Gatling website which will help you write simple HTTP calls to your webservices but it gets bit difficult to incorporate the exact scenario of your test case because there are not much variety of example code which you can quickly edit and use. In this article I will try to show most of the structural construct that are frequently used while making a series of dependent web-service calls, typically used by a mobile game client while interacting with the game server.

Setup instructions are well described in the getting started tutorial. Following scala code will make a POST request to a mock REST endpoint that I have created using mockable.io.

Simple Request



 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


package package1

import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._
import io.gatling.jsonpath._

class Test1 extends Simulation {

  val httpConf = http
    .baseURL("http://demo1263864.mockable.io/") 
    .acceptHeader("application/json") 

  val scn = scenario("Simple scenario")
    .exec(http("First Request")
      .post("gatling")
      .body("""{"data":1}""")
      .headers(headers_1).check(jsonPath("$.result").is("SUCCESS")))

  setUp(scn.inject(atOnceUsers(1)).protocols(httpConf))
}


Above code just sends following request
POST http://demo1263864.mockable.io/gatling
with request body as
{"data":1}
I have kept the request body very simple as it does not have any effect. My REST services always responds with a fixed response, that is


1
2
3
4
5
6
7
8


{
  "result": "SUCCESS",
  "data": [
    {
      "score": 200
    }
  ]
}



Note that jsonPath("$.result") fetches the top level JSON attributes and checks if it's value is "RESULT", otherwise it shows as one failure ("KO") in the final report. We could also check the HTTL status using .check(status.is(200))) . More about check().

Custom Headers and Body


1


  val headers_1 = scala.collection.mutable.Map("Content-Type" -> "application/json")


Notice that I have used a mutable map to initialize the header and later converted to a Map (immutable) because sometimes output of one requests might have to be input in the second request as part of request header or body. e.g.


1
2
3
4
5
6


  val scn = scenario("Simple scenario")
    .exec(http("First Request")
      .get("gatling").headers(headers_1.toMap)
      .check(jsonPath("$.result").is("SUCCESS"), jsonPath("$.data[0].score").saveAs("score")))
    .exec(http("Second Request").post("gatling1").headers((headers_1 += "score" -> "${score}").toMap)
      .body(StringBody("""{"data":"${score}"}""")).check(status.is(200)))



In this case the attribute "score" is fetched from the output of the first request and then fed to the header and body of the second request. .saveAs("score") saves the value of score in the session with key "score" and makes it available to be fetched later. The session separate for all users that are injected by the setup(). So all user specific data which does not need to be shared across users should be saved in the session. "${score}" is an EL expression used for retrieving the value saved earlier in the session. More about session.

Looping

Let's say our output JSON array is as follows , which contains an array which we need to loop over.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


{
  "result": "SUCCESS",
  "data": [
    {
      "playerId": 1,
      "score": 200
    },
    {
      "playerId": 2,
      "score": 300
    },
    {
      "playerId": 3,
      "score": 400
    }
  ]
}


Our code to consume it and iterate over all the player/score entry would be


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


  val scn = scenario("Simple scenario")
    .exec(http("First Request")
      .get("gatling")
      .headers(headers_1.toMap)
      .check(jsonPath("$.result").is("SUCCESS"), jsonPath("$.data[*]").ofType[Map[String, Any]].findAll.saveAs("pList")))
    .foreach("${pList}", "player") {
      exec(session => {
        val playerMap = session("player").as[Map[String, Any]]
        val playerId = playerMap("playerId")
        session.set("playerId", playerId)
      })
    }



In above code snippet we are looping over a JSON array which output of the earlier request. pList stores the all the entries in the JSON array temporarily in the session and then .foreach block loops over it feeding one at a time with the session key "player". In this loop at the end we save the required value with session.set method. Since session is immutable all the session manipulation code wrapped around a session function as above.

Notice that "$.data[*]" is jsonPath expression to get hold of all the elements of the array attribute "data". More about jsonPath expressions.


We can also use a counter to loop over a list and fetch corresponding element from another list when more than one list is saved in the session. This is useful when the JSON structure is nested. Refer the following JSON structure.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


{
  "result": "SUCCESS",
  "data": [
    {
      "playerId": {
        "id": 2,
        "version": 1
      },
      "score": 200
    }
  ]
}




 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


  val scn = scenario("Simple scenario")
    .exec(http("First Request")
      .get("gatling")
      .headers(headers_1.toMap)
      .check(jsonPath("$.result").is("SUCCESS"),
        jsonPath("$.data[*].playerId.id").findAll.dontValidate.saveAs("ids"), jsonPath("$.data[*].playerId.version").findAll.dontValidate.saveAs("versions")))
    .exec(session => {
      val ids = session("ids").as[Vector[String]]
      val versions = session("versions").as[Vector[String]]
      val validIds = new ListBuffer[String]()
      ids.indices.foreach { i =>
        if (versions(i) == "0") {
          validIds += opptetrisIds(i)
        }
      }
      session.set("validIds", validIds.toList)
    })


In above code we save both the Ids and corresponding versions in separate list and then loop over the ids. We create a new list of IDs for which the corresponding version is 0. Finally we save the filtered Ids for further processing.

Flow branching


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34


  //Global Users
  val userMap = scala.collection.concurrent.TrieMap[String, String]()
  //Randomiser 
  val rnd = new scala.util.Random


  val headers_1 = scala.collection.mutable.Map("Content-Type" -> "application/json")

  val scn = scenario("Simple scenario")
  exec(session => {
    var newUser = (rnd.nextInt(10) > 8 || userMap.size < 1)
    session.set("newUser", newUser)
  })
    .doIfOrElse(session => session("newUser").as[Boolean]) {
      exec(
        http("If request")
          .post("gatling")
          .headers(headers_1.toMap)
          .body(StringBody("""{}""")).asJSON
          .check(status.is(200)))
    } {
      exec(session => {
        val index = rnd.nextInt(userMap.size)
        val userArray = userMap.toArray.map(x => Array(x._1, x._2))

        session.set("id", userArray(index)(0))
          .set("version", userArray(index)(1))
      })
        .exec(http("Else Users")
          .post("gatling")
          .headers(headers_1.toMap)
          .body(StringBody("""{"id":"${id}, "version":"${version}"}""")).asJSON
          )
    }




In the above code we maintain a global map. Let's assume we need a pool users who will be making requests to the server and we also need the user to be an existing user sometimes and a brand new users otherwise. Say 2 in 10 users should be new and rest should be taken from the existing pool. We maintain a global userMap where we store all the newly created users and we refer the same map to fetch existing users. The global map is visible to all the threads so its not specific to any session.

Finally the load test plan is the most important part of the whole activity where you should be able to control the arrival of the users. e.g. if you want 10 users to arrive in uniform rate over the span of 30 seconds following should be your injection code.


1
2


  setUp(
    scn.inject(rampUsers(10) over (30 seconds)).protocols(httpConf)))


There are several other ways to create the desired pattern in this tutorial.

References

Sunday, July 20, 2014

Its all over ! - FIFA 2014

The FIFA world cup was pretty much over when German Mario Goetze received the ball on his chest and kicked it inside the Argentine goal post. The world cup season of 1998, 2002 , 2006 and 2010 have been special (be it good or bad) for me , not sure what was special personally this time but I followed almost every match this time.

In India Sony SIX broadcasted this event but it started with a very disappointing manner given the fact that John Abraham was in the analyst panel in first few matches. He had done some great movies but no way qualified to be there. Actors should be called to these event only if they paid a lot to do their film promotion :).

Apart from cable TV the XBMC plugin Sports Devil played a very important role in live streaming the matches. Both ESPN and BBC 5 Live were airing the audio commentary which was a live saver when were driving during matches.

Friday, July 18, 2014

Media features

To kill the daily commute time in city traffic I listen to couple of podcasts. While Indicast is release once a month on an average, BBC World Update : Daily commute is published everyday with news summery everyday.

Last week I sent an audio captured in my village to BBC and featured it in their "World Update" program which is hosted by Dan Damon. You can listen to the episode, you can seek to 40 mins.
Here is the actual clip published by them.



Previously I sent a picture to the Indicast which got featured in the Episode 197 Check out the last 4:05 minutes.

 

Finally NewsLaundry featured a clip from Aaj Tak channel I sent. Check out this episode at 23:02.



Earlier Clothesline featured one more clip pointed by me and Madhu Trehan was generous enough to give me credit also :)