Using AWS Lambda we are able to run micro-service sized applications without any traditional server infastruture. These are called "serverless functions", which run inside ephemeral containers that offer several advantages over the traditional server/host configuration.

We've used AWS Lambda functions for some time now. They're a great way to perform a simple computing at a large scale. And the best part - it comes without the need to manage server instances and other resources.

Lambda allows you to write functions in a variety of different langauges (Java, Node.js, C# and Python). Our functions run on NodeJS.

The latest function we're working on has a single third-party library dependency. This adds an additional step to the development process, since we can't simply paste the function code into the inline editor for testing and deployment. The application package must be manually created via CLI - yet after doing that 2 or 3 times it starts to become redundant. Run a command, wait for it to complete, then run the next and so on...

It was time to automate this process. The solution was to use Make. It's super easy to setup, it's available on Mac and it's grown-up software.

Making things

Our packaging process has the following steps:

1. Remove any old package.
2. Install Node non-dev dependencies.
3. Package the function and dependencies together.

Our Makefile is almost an exact representation of that recipe:

lambda.zip: clean
    rm -rf node_modules/
    npm install --only=production
    zip -r $@ lambda.js node_modules/

clean:
    rm *.zip

It's possible to have several "rules" to make a target in your file. Here, we have two targets: lambda.zip and clean. Simply running make – without params – from the command line will build the ZIP package. However running make clean will just remove all the ZIP files in the directory.

Our lambda.zip target depends on clean to run. But clean depends on nothing. Multiple dependencies are also allowed, if needed. If that's the case, separate them using spaces.

Below the target and its dependencies, there are the commands used to build the target. They must be indented using a tab. The $@ is a reference to the target name.

Going further

We have a third target to test the app. It runs the test command declared in the package.json file.

test:
    rm -rf node_modules/
    npm install
    npm test

Notice the difference in the dependencies install? Here we add all packages, not just the non-dev. That's because we have some libraries that are used for testing purposes.

Since we removed the node_modules directory in both lambda.zip and test targets, we could move it to a separated target and add it as a dependency. We ended up not doing that because it's a single liner and we don't use it alone. In contrast with clean, which is occasionally used with make clean.

Conclusion

This post demonstrates a very simple way to use Make in your development workflow. Its powerful, yet flexible and there are people using it to build far more complex recipes.

If you want to dig deeper and learn more about Make and Makefile structure, there are lots of resources on the web. Though, you'll quickly find articles are about compiling C/C++ programs. This Digital Ocean article is a good starting place to learn how Make works and how it can be used to automate tasks. The man page is also handy, especially after you have some experience.

Thanks for reading! If you have any comments or questions please leave them below.

We recently ran into a situation where the business logic of a project dictated that we can not have duplicate file contents. Regardless the naming convention, the contents of each file needed to remain unique. In this scenario, we were storing all types of uploaded files in a single directory.

To prevent content duplication, we needed to compare the newly uploaded file with files already in the destination directory.

Unfortunately, the Laravel documentation is not clear about the uploaded file class. Digging into the Laravel source code, you may find that instances returned by the file() method (from the request) are inherited from SplFileInfo. This means it is possible to manipulate the uploaded file with little hassle. Better yet, we can do this before storing the file.

You may find the evidence of the SplFileInfo class in File. It extends that class:

class File extends \SplFileInfo  
{

This reference to Symfony code may look odd if you are not familiar with the Laravel architecture, however Laravel uses lots of Symfony's components to form it's foundation.

Lets dive in and see how this class can be leveraged in our apps.

Handle upload

A basic file upload action may get the file instance from the request, check if an actual file was uploaded and finally store it. That may look something like this:

<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;  
use Symfony\Component\HttpFoundation\Response;

class FileController extends Controller  
{

    /**
     * Handle the file upload.
     *
     * @param  Request $request The request instance.
     * @return Response A response instance.
     */
    public function upload(Request $request)
    {

        /*
         * The form field was named "file".
         */
        $file = $request->file('file');

        if (empty($file)) {
            abort(400, 'No file was uploaded.');
        }

        $path = $file->store('uploads');

        // Do whatever you have to with the file.

    }
}

Read and compare file chunks

Using the aforementioned file instance, we may write a function to compare the file with another SpfFileInfo instance. To speed things up, we don't have to compare the entire file. Its possible to read a small chunk of same size from each file and check the differences between them. As soon as a difference is found, the method returns.

/**
 * Compare the contents of two files to check if they have some difference.
 *
 * It does not check what exactly is different between files. This way, as 
 * soon as a difference is found, the function stops reading the files.
 *
 * @param  SplFileInfo $a The first file to compare.
 * @param  SplFileInfo $b The second file to compare.
 * @return bool Indicates if the files have differences between them.
 */
private function fileDiff($a, $b)  
{
    $diff = false;
    $fa = $a->openFile();
    $fb = $b->openFile();

    /*
     * Read the same amount from each file. Breaks the loop as soon as a 
     * difference is found.
     */
    while (!$fa->eof() && !$fb->eof()) {
        if ($fa->fread(4096) !== $fb->fread(4096)) {
            $diff = true;
            break;
        }
    }

    /*
     * Just one of the files ended. This is unlikely to happen, though. 
     * Since we already checked before if the files have the same size.
     */
    if ($fa->eof() !== $fb->eof()) {
        $diff = true;
    }

    /*
     * Closing handlers.
     */
    $fa = null;
    $fb = null;

    return $diff;
}

What about performance?

Our function above is a good way of comparing two files contents. However, opening, reading and comparing the contents of every file in our uploads directory will be far too resource consuming (time, I/O, processing). We can significantly reduce the performance overhead by comparing the file sizes first, and only checking the content difference of matching size files. Here is what that would look like:

/**
 * Check if the given file was already uploaded.
 *
 * It loops through all files in the uploads directory, looking for anyone that 
 * could be equal to the currently uploaded file.
 *
 * @param  SplFileInfo $file The file to check.
 * @return bool Indicates if the file was already upload.
 */
private function isAlreadyUploaded($file)  
{
    $size = $file->getSize();

    /*
     * The directory where the files are stored.
     */
    $path = storage_path('app/uploads/');

    if (!is_dir($path)) {
        return false;
    }

    $files = scandir($path);
    foreach ($files as $f) {
        $filePath = $path . $f;
        if (!is_file($filePath)) {
            continue;
        }

        /*
         * If both files have the same size, check their contents.
         */
        if (filesize($filePath) === $size) {

            /*
             * Check if there are differences using the function we wrote above.
             */
            $diff = $this->fileDiff(new \SplFileInfo($filePath), $file);

            /*
             * Return the files are not different, meaning equal, that is 
             * already uploaded.
             */
            return !$diff;
        }
    }
    return false;
}

Handle and store file

Finally, we can call the last method from the upload action. Remember to only store the file after checking if it was previously uploaded.

/**
 * Handle the file upload.
 *
 * @param  Request $request The request instance.
 * @return Response A response instance.
 */
public function upload(Request $request)  
{

    /*
     * The form field was named "file".
     */
    $file = $request->file('file');

    if (empty($file)) {
        abort(400, 'No file was uploaded.');
    }

    /*
     * There is a file equal to the currently uploaded one in the uploads dir?
     */
    if ($this->isAlreadyUploaded($file)) {
        abort(400, 'This file was already imported.');
    }

    /*
     * Only stores the file after successfully checking if it is not at the 
     * destination already.
     */
    $path = $file->store('uploads');

    // Do whatever you have to with the file.

}

Conclusion

This is just one way to take advantage of the file roots to manipulate it. There are probably lots of other use cases where you may leverage this.

Also, its important to point out that this is just a helper given by the framework. You can certainly take other approaches to achieve the same result. For instance, its possible to store the file to a temporary path and then run all the required checks. Or use other file-related functions from PHP to manipulate the uploaded file.

Thanks for reading! If you have any comments or questions please leave them below.

Traditionally, many developers have opted to use relational databases when dealing with complex relationships between objects. For those of you who are used to working with relational databases like MySQL, a NoSQL database like Redis may seems like a difficult transition.

While Redis is best suited for use cases like cache, queues, and leaderboards - it is possible to use Redis in front of a MySQL database. Doing this can improve overall query performance, while still allowing you to execute complex queries.

Let's go ahead and dive deeper to see what can be achieved using Redis based queries.

Storing Object Data

Unlike traditional (RDBMS) databases, Redis is a key-value store. But it's an advanced one. There are many ways to store the values beyond a simple key/value pair, like lists or hashes. The way you choose to initially store the value will affect what you can do with it later.

The methods we've listed below is not intended to be comprehensive. It's intended to just be a starting point.

Set (SADD)

Within Redis, a set is a list of values. It may be used when a value is shared by multiple objects.

To store a member in a set, use SADD:

> SADD key member

Multiple members can be stored in a single call. We can also store product categories using a set, since several products will be under the same category.

SADD can be called this way:

> SADD product:category:1 5 10

With this command, products with an ID 5 and 10 will be stored under the key of category with an ID 1. It may seem odd to use the value (category ID) as the key. However, as you'll see in a bit, this is the best approach.

Sorted Set (ZADD)

If you plan to query a value using ranges or comparison conditions such as less than or greater than, your pick is the sorted set.

The command used to store sorted set members is ZADD:

> ZADD key score member

You may pass multiple score/member pairs, if you wish to.

Sorted set are great to store information like prices. For this situation, the command will look something like:

> ZADD product:price 49.99 1 54.90 2

We are storing the price (score) 49.90 for the product with ID 1, and 54.90 for the product with ID 2. Both are set under the product:price key.

Hash (HSET and HMSET)

When each entity has a different value for an attribute, but you still want to query it, hash is a good choice.

Call HSET to set a field using a value:

> HSET key field value

Note: Unlike the ZADD and SADD commands, HSET doesn't allow you to store multiple members (field and value) in a single call. Fortunately you can use a seperate but related command achieve that: HMSET.

Say you would like to search for products by their SKU. Since each product has a unique SKU, you can use a hash to store them by doing this:

> HSET product:sku '1ZN820' 1

Doing that, you're storing the SKU 1ZN820 for the product with an ID 1 within product:name key.

Querying Object Data

For each storage type (set, sorted set and hash) there are specific commands to fetch the keys and values. That is why its important to choose your storage method carefully, as you will be limited by the query on the other end.

Querying Sets (SMEMBERS)

Starting with the set type and the example given above, if you want to retrieve all products in a specific category - let's use the category with ID 2 - you simply call:

> SMEMBERS product:category:2

That will return a list of product ID's. With those ID's you can retrieve the product's objects.

In our specific use case, the full objects were stored in MySQL and we used Redis to speed up querying. We were doing this by performing a SELECT query on MySQL:

WHERE products.id IN ( ... ).

Ultimately, it's up to you to decide where to store the objects: in Redis or elsewhere. Each use case is different.

Querying Sorted Sets (ZRANGEBYSCORE)

Previously, we saw that sorted set is a good choice to store values we want to query as numeric values – by range or using comparisons. This is not the only use for sorted sets, but its the only one we'll cover.

If you want to retrieve all product with price greater or equal to 50, you may call:

> ZRANGEBYSCORE product:price 50 +INF

The command expects 3 arguments: the key, lower limit and upper limit. As you can see we're passing the positive infinite (+INF) as the third argument. The negative infinite (-INF) is also available if you need it.

A specific range, like 40-50, can be also be used:

> ZRANGEBYSCORE product:price 40 50

Querying Hashes (HMGET and HSCAN)

Finally, to search within the strings stored under a hset, let's perform the following call:

> HMGET product:name 'Cool product'

This would return the product having that exact name. It's also possible to do a partial match, similar to SQL's LIKE operator:

> HSCAN product:name 0 MATCH *ol*

The * is a wildcard for any amount of any characters. The command above would match cool, toll and pole.

You probably noticed the 0 as second argument. That is the current cursor. All Redis' commands from the SCAN family are iterators based on a cursor. After each call, the server will return the cursor for the next iteration. You should use it in the next call. Keep calling the server until you get 0 in the cursor.

Using Lua for SCAN iteration

There is no command in Redis to loop through an iterator. Fortunately, Redis supports scripting with Lua and we can use a small script to achieve that:

    local cursor = 0
    local fields = {}
    local ids = {}
    local key = 'product:name'
    local value = '*' .. ARGV[1] .. '*'

    repeat
        local result = redis.call('HSCAN', key, cursor, 'MATCH', value)
        cursor = tonumber(result[1])
        fields = result[2]
        for i, id in ipairs(fields) do
            if i % 2 == 0 then
                ids[#ids + 1] = id
                redis.call('SADD', KEYS[1], id)
            end
        end
    until cursor == 0
    return ids

The code above returns all products that match a given value. It also stores that same result in a set, so we can use it again later.

To parse and run the script, call:

> EVAL 'local cursor; [...]' 1 tmp:name cool

EVAL expects a script as the first argument, then the number of keys we're going to pass to the script, followed by the list of keys, and finally the list of arguments.

In our case here we passed one key (tmp:name) and one argument (cool). Inside the script, the key is accessed with KEYS[1] and the argument is accessed using ARGV[1].

Putting all together

By now, you have seen how to store and query each type of data. It's really useful, but in some cases the querying rules are more complex.

Let's say you have to find all products whose name contains "cool" and are priced in 50.00 or more. One way to do this is to fetch ID's for each condition and intersect the results client-side. However, in our opinion there is a better way:

> ZUNIONSTORE tmp:price 1 product:price WEIGHTS 1
> ZREMRANGEBYSCORE tmp:price -INF (50
> EVAL 'local cursor; [...]' 1 tmp:name cool
> ZINTERSTORE tmp:result tmp:price tmp:name
> ZRANGE tmp:result 0 -1

Let's go through each call:

1. We start by creating a temporary copy of the product:price key using ZUNIONSTORE.
2. We remove all values lower than 50 from the temporary copy. The ( before the number indicates that 50 must be kept in the set.
3. Run the iterator script storing all matched ID's in a temporary key.
4. Intersect all values in tmp:price and tmp:name, storing the result as another key.
5. Finally, we get all the values from the previous intersection.

The last call will give you a list of ID's that, as explained in the begin, can be used to retrieve full objects.

Getting all products whose name contains "Cool" or are priced in 50.00 or more, is as simple as replacing ZINTERSTORE with ZUNIONSTORE in the fourth call.

Conclusion

It is possible to combine more unions (OR) and intersections (AND) to fulfill even more complex logic. However, keep in mind that when mixing those group operations, the order does matter.

To dive into further detail on each command, the Redis list of commands is a great place to start.

We have used Cordova + Ionic in the past to quickly prototype mobile apps for clients. Its proved to be a valuable tool because it allows us to re-use existing HTML, CSS and JavaScript we may have from a web project and compile it directly to iOS, Android and other native platforms.

With the release of the Apple Watch, we were curious to see if there was a way to quickly prototype apps for the watch, using Cordova. It turns out you can, with the help of Cordova Apple Watch from Telerik.

This is part of a 2 part post series. Between our two posts, we are going to build a simple navigation app for the iPhone and Watch, which work harmoniously together. In this first post we are working on just the iPhone app, which will allow users to search for an address, which is validated via Google Maps and initialize direction data.

Set up Cordova + Xcode + Apple Watch Plugin

First, lets create a new Cordova project. Be sure to have Ionic CLI tools installed on your machine. We will need to install the xcode package from npm - the plugin hook will need it to inject code into the Cordova project.

$ ionic start locator blank
$ cd locator
$ npm install xcode
$ ionic platform rm ios

As of this post, the latest version available is Cordova iOS v4.1.0. However, the Apple Watch plugin from Telerik is not yet compatible with Cordova iOS 4+, so we will need to install Cordova iOS 3.9.2 to get it working.

Before installing the iOS platform, open locator/config.xml and add your app ID - mine is co.kettleio.demo.locator. Your config.xml should look something like this:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>  
<widget id="co.kettleio.demo.locator" version="0.0.1" xmlns="http://www.w3.org/ns/widgets" xmlns:cdv="http://cordova.apache.org/ns/1.0">  
  <name>locator</name>
  <description>
        An Ionic Framework and Cordova project.
    </description>
  <author email="you@example.com" href="http://example.com.com/">
      Your Name Here
    </author>
  <content src="index.html"/>
  <access origin="*"/>
  <preference name="webviewbounce" value="false"/>
  <preference name="UIWebViewBounce" value="false"/>
  <preference name="DisallowOverscroll" value="true"/>
  <preference name="SplashScreenDelay" value="2000"/>
  <preference name="FadeSplashScreenDuration" value="2000"/>
  <preference name="android-minSdkVersion" value="16"/>
  <preference name="BackupWebStorage" value="none"/>
  <feature name="StatusBar">
    <param name="ios-package" value="CDVStatusBar" onload="true"/>
  </feature>
</widget>  

Next, lets add the iOS platform to the project by running the following command:

$ ionic platform add ios@3.9.2

With most of the preliminary work out of the way, we can begin working directly on the Xcode project. Open up the Xcode project, which is located in platforms/ios and make the following updates:

1. Go to File > New > Target > watchOS > Application > Watchkit App for watchOS 1
2. Select Objective-C as the language.
3. Check the boxes for Glance and Notifications
4. Click on Finish and then click on Activate

Now let's switch back to the terminal and install the Telerik plugin:

$ ionic plugin add https://github.com/Telerik-Verified-Plugins/AppleWatch

If the plugin was installed correctly you will see an Apple Watch drawn with ASCII and a few other messages indicating header files were properly installed.

Configure Xcode

There are a few more steps necessary to build the project correctly.

Bundle Version

Set CFBundleVersion (bundle version) and CFBundleShortVersionString (bundle versions string, short) of all targets to the same value. To do this, use XCode's search feature COMMAND + SHIFT + F and change all 3 .plist values. This value must match your config.xml file, so in this example I'm setting the version to 0.0.1.

Register App ID

Register the app ID in your Apple Developer Account: Certificates, Identifiers & Profiles > App ID's > New.

Ensure this app ID matches the one we set inside config.xml earlier. You will also need to select App Groups in the enabled services section at the bottom.

Register App Group

We need to register an App Group in our Apple Developer Account. From the dashboard go to Certificates, Identifiers & Profiles > Identifiers > App Groups > Add New.

Create + Assign Provisioning Profile

1. Go to Provisioning Profiles > Development and create a new profile for iOS App Development.
2. Download the new profile and open it in Xcode.
3. Go to the targets (Capabilities tab) and add the newly created app group to your target.
4. Navigate to the Build settings tab and search for Bitcode. We will disable it by toggling the option to NO.
5. Repeat steps 3-4 for all three targets:

Note re: -lmmwormhole

Some users have experienced this error:

ld: library not found for -lmmwormhole  

You can fix this by searching the Frameworks directory within Xcode for "libmmwormhole.a". There will be an option for Target Membership on the right sidebar of Xcode.

Deselect both targets, select them again, clean up the project ( Product > Clean) and then build the project again. This has seemingly fixed the linking issue.

Create Routes, Controllers & Views

Now that we have a baseline up and running, lets recap the project goals:

1. Create a view that will allow us to search for a valid location within our iOS app.
2. Create a view that will show us the compass and destination data. This data/view will eventually be passed to the watch.

This means we need to create two distinct views within our app: map and compass. Before we start building them, we are going to install 2 plugins. They will help us quickly build out the compass functionality:

$ ionic plugin add cordova-plugin-device-orientation
$ ionic plugin add cordova-plugin-geolocation

Setup the Map

We will start by creating a route and controller for the map. Open up your www/js/app.js file and add the following route:

.config(function($stateProvider, $urlRouterProvider) {

  $urlRouterProvider.otherwise('/')

  $stateProvider
  .state('index', {
    url: '/',
    templateUrl: 'map.html',
    controller: 'mapCtrl'
  })

})

Next, with the help of the Google Maps API, we are going to build the controller for our map view. The code we are going to modify can be found on Google's developer site.

With a few tweaks, our map view controller will look like this gist. This code should go after your routes.

Some important notes about this block of code:

1. The controller starts with $scope.ifValidAddress = false - this will allow us to keep the compass hidden, until we find a valid location on the map.
2. The $scope.disableTap function is responsible for the Google Maps autocomplete.
3. We created a Factory to save the search location. This will allow us to share the location information between views.

Map View

The map view will simply have a fullscreen Google Map with a text input for location searching. We will connect the search to the Google Places API for suggestions. Open the index.html file and include the Google Maps API & Google Places libraries:

<script src="https://maps.googleapis.com/maps/api/js?key=YOUR_API_KEY&libraries=places" async defer></script>  

Note, you will need to add your own API Key from the Google Maps API.

You will also need to update the <body> of your index.html file so it looks like this:

<body ng-app="starter">

 <ion-nav-bar class="bar-positive">
   <ion-nav-back-button>
   </ion-nav-back-button>
 </ion-nav-bar>
 <ion-nav-view></ion-nav-view>

</body>  

Next, we are going to create a new file in the www directory called map.html and add our map view to it. The view will look like this:

<ion-view view-title="Map">

   <ion-nav-buttons side="right">
      <button class="button icon-right ion-compass button-clear" ng-click="openCompass()" ng-show="ifValidAddress"></button>
   </ion-nav-buttons>
    <ion-content scroll="false" >

      <input id="search-place" class="controls" type="text" placeholder="Search Address" ng-focus="disableTap()">
      <div id="map"></div>

    </ion-content>

</ion-view>  

This view accomplishes a few things for us:

1. We are going to conditionally show the compass button if a valid location is found. We are using the ng-show="ifValidAddress" function for this.
2. We added an input with id="search-place". We'll use this data input to query the Google Places library.
3. We added a div id="map" to insert the Google map.

Map Styles

Now lets make it sexy with some CSS. Place the following code inside www/css/style.css:

#map {
    height: 100%;
}

.controls {
    margin-top: 10px;
    border: 1px solid transparent;
    border-radius: 2px 0 0 2px;
    box-sizing: border-box;
    -moz-box-sizing: border-box;
    height: 32px;
    outline: none;
    box-shadow: 0 2px 6px rgba(0, 0, 0, 0.3);
}

#search-place {
    background-color: #fff;
    font-family: Roboto;
    font-size: 15px;
    font-weight: 300;
    margin-left: 12px;
    padding: 0 11px 0 13px;
    text-overflow: ellipsis;
    width: 200px;
}

#search-place:focus {
    border-color: #4d90fe;
}

#type-selector {
    color: #fff;
    background-color: #4d90fe;
    padding: 5px 11px 0px 11px;
}

#type-selector label {
    font-size: 13px;
    font-weight: 300;
}

Test It

At this point, you should be able to preview the app in your browser by running ionic server -lab. If the view is rending correctly and we are integrated with the Google Places API, you should see a map and search box:

Setup the Compass

Our final step is to create the compass view, which will show us all the navigation details. We are going to include the following details:

1. Distance to destination (in meters)
2. Bearing to destination (in degrees)
3. Current heading
4. Different between current heading and destination bearing (in degrees)
5. Render a compass arrow based on the difference between the current heading and the destination bearing

Add Route

Open the app.js file and add a route for your compass:

  .state('compass', {
    url: '/compass',
    templateUrl: 'compass.html',
    controller: 'compassCtrl'
  });

Create View

Create a new file compass.html and add the following:

<ion-view view-title="Compass">

  <ion-content>

    <div class="list" id="results">
      <div class="item item-divider">
        Location Details
      </div>

      <div class="item">
        Distance to destination
        <span class="item-note">
          {{ distance }} meters
        </span>
      </div>

      <div class="item">
        Bearing to destination
        <span class="item-note">
          {{ bearing }} degrees
        </span>
      </div>

      <div class="item">
        Current heading
        <span class="item-note">
          {{heading}} 
        </span>
      </div>

      <div class="item">
        Difference in heading and bearing:
        <span class="item-note">
          {{difference}} degrees
        </span>
      </div>

    </div>

    <div class="card">
      <div class="item item-divider">
        Compass
      </div>
      <div class="item item-text-wrap arrow-container">
        <div id="arrow" style="{{ animateRotation }}"></div>
      </div>
    </div>

  </ion-content>

</ion-view>  

Update index.html

We are going to include a library by Chris Veness to help manipulate the coordinates. Add this file and include it inside your index.html:

<script type="text/javascript" src="js/latlon.js"></script>  

Create Controller

With a view created, we must setup the controller which will look something like this gist.

Conclusion

At this point, you should be able to search for an address on your phone. Once a valid destination is found, you can tap the compass icon in the top right corner and watch the data update in real-time.

As we mentioned in the beginning, this is part one of our two part series. We will be posting a follow up in the coming weeks that outlines the Watch functionality.

If you'd like to view the entire code it is available on our GitHub. As always, comments are welcomed - please let us know if you found this helpful or have any questions!

Quite often we, as developers, cache resources in memory to improve app performance and reduce server load. By storing frequently accessed data in memory you avoid using CPU cycles to fetch a fixed set of data from your database.

Traditionally the cache concept has been limited to static data or data that has a long lifespan. What about API's that serve up frequently changing data? Many times this data is not truly "dynamic", but rather constantly changing. It's possible to cache these type of requests using an event driven architecture.

Event Driven Scenario

The concept is simple - every time a data change occurs an event is triggered. This event forces your cache to flush the old data and insert the new data.

Lets use the example of an eCommerce site with inventory data that contains product SKU's, price and number of units available. This data is cached so various services, like the website and iOS app, can pull product inventory levels without putting a tremendous load on the server.

Every time a new unit is sold or additional stock is added, the cache for that product must be updated. In this example, we have made the assumption that you are using a traditional database (MySQL or Postgres) with a cache layer placed on top. It is critically important to flush and insert the new data as quickly as possible to avoid any kind of read errors on inbound requests. Given the requirement of a quick flush and insert, what caching software is the best choice?

Redis or Memcached

When it comes to in-memory caching there are two major players: Redis and Memcached. Both are key-value storage based on NoSQL architecture.

Our simple experiment is not intended to represent all use cases, but rather provide some kind of baseline. Which software will not only be able to flush data quickly, but also re-populate it quickly?

To compare the two we generated a sample data set with 500,000 records of SKU's, prices and units available. We inserted the entire data set into the cache, then flushed it. We ran this test 5 times and compiled the results below:

Redis (500,000 records)

Insert:  
        Max: 24196 ms
        Min: 23817 ms
        Avg: 24014 ms
Flush:  
        Max: 177 ms
        Min: 175 ms
        Avg: 176 ms

Memcached (500,000 records)

Insert:  
        Max: 36862 ms
        Min: 36041 ms
        Avg: 36442 ms
Flush:  
        Max: 1 ms
        Min: 0 ms
        Avg: 0 ms

We expected Redis to win outright on both tests. However, the results reveal that Memcached has near instant flush characteristics. The insert performance of Redis was significantly better than Memcached, doing it nearly 35% quicker. With a larger data set, it's likely this margin would grow further.

This analysis does not account for the actual performance of the cache when serving requests. Generally Redis is regarded as having better performance and more robust caching due to a more advanced memory management algorithm. This experiment was simply intended to benchmark the insert/flush speed of each software.

Please leave a comment below if you have any questions!

Just recently we've begun running tests with a new language called Elixir and we are very excited about it for a few reasons. The language is built on top of the Erlang VM & Open Telecom Platform (OTP), which is a highly proven network language that is used extensively in telecommunications. It also forms the foundation of popular apps like Facebook Chat and WhatsApp.

So why are we excited about this language here at Kettle? We see a few high level benefits that it brings to the web:

Concurrency: Elixir is designed around what they call a shared-nothing actor model, which allows it to efficiently take advantage of every available CPU core.
Hot Code Updates: rather than immediately killing pending requests during code updates, Elixir will gracefully handle these processes before it updates the code.
Fault-Tolerant: Elixir has built-in supervisors that restart parts of your program by reverting to a state that was known to be working.

Introducing Phoenix

As you can see, Elixir is a great choice when you need an extremely reliable and fast network application. There is a fantastic framework designed on top of Elixir called Phoenix. It ships with many of the essential tools needed for building a web application:

• Familiar MVC structure making it easy and pleasant to work with
• Database drivers via Ecto
• HTTP server via Cowboy
• Module for filtering requests and responses via Plug
• Various tools for package management (hex) and unit testing (ex_unit)
  
  

Building a Scoring API

Today we want to give Elixir a try and build a simple application with it. Our goal is to build an API that streams player scoring data, to a leaderboard, in real-time. In this example we are going to use MySQL, but if you're looking for something über fast, we recommend using MongoDB, which also has a native database driver included in Phoenix.

For this example, we will be using Phoenix V1.0.3.

Install Phoenix

There is an awesome getting started guide on the Phoenix website and we recommend following that. Here is a quick rundown of what you need to do. Make sure you have Homebrew on Mac installed first. Then install the prerequisite software:

$ brew install elixir
$ brew install erlang
$ brew install node
$ mix local.hex

After that, you can begin to install the phoenix framework:

$ mix archive.install https://github.com/phoenixframework/phoenix/releases/download/v1.0.3/phoenix_new-1.0.3.ez
$ mix phoenix.new kettle_phoenix --database mysql 
## Note the database flag which defines MySQL as our DB driver
$ cd kettle_phoenix
$ mix ecto.create
$ mix phoenix.server

You should now see the Phoenix welcome screen at http://localhost:4000.

Create API Resources

Traditionally you would have to go through the process of creating a model, view and controller for your API to render. However, in Elixir there is a simple command to automate this process. It's called Mix.Tasks.Phoenix.Gen.Json and it will automatically create the following resources for you:

• model in web/models
• view in web/views
• controller in web/controllers
• migration file for the repository
• test files for generated model and controller

Let's go ahead and run this command to create the necessary resources to render our player names and their scores in a JSON file:

$ mix phoenix.gen.json Score scores player:string score:integer

Add Routes

The terminal output from this command will return a new resource for the API scope, which belongs in web/router.ex. If you modified the data structure in the command above you will have a different output, however for this project we'll update our file to look like this:

defmodule KettlePhoenix.Router do  
  use KettlePhoenix.Web, :router

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/api", KettlePhoenix do
    pipe_through :api

    resources "/scores", ScoreController, except: [:new, :edit]
  end 

end  

Connect Database

Next, we need to update the database connection details in config/dev.exs.

Then you can run the following commands to run the pending migrations:

$ mix ecto.migrate

At this point we should verify the database has been updated. If everything looks good we can seed some data into the database.

Go ahead and start up the Elixir server:

$ mix phoenix.server

We should see the endpoint serving up all player data at: http://localhost:4000/api/scores. If you'd like to insert data you can do that via a simple cURL call:

$ curl -X POST -H "Content-Type: application/json" -d '{ "score": { "player": "Jack Bauer", "score": "100" } }' http://localhost:4000/api/scores

Conclusion

While this was a quick and simple example, it should highlight how developer friendly Elixir is. With a relatively small amount of code we are able to build a very robust, scalable and fault-tolerant API. You could certainly add additional routes, with filters to sort and parse this data further.

We hope you enjoyed this article. Please feel free to leave comments below. The entire project is available on Kettle's GitHub.

cURL is a great library and tool. Quite often we'll use it to interact and fetch data from third-party API's. However, sometimes debugging requests made with cURL, specifically from a PHP application, can be tricky.

Let's say you have an integration with a third-party API and the service becomes unstable. Some requests respond as expected, while others do not. How can we determine where the code is failing?

Use Logs, Not Exceptions

Just recently, we ran into a situation where one of our cURL integrations was failing. In production environments we capture and log all exceptions using an app called Sentry. Unfortunately, in this particular situation, the exception output did not reveal very much about what was happening with our cURL requests.

To get detailed information we needed to save detailed information to a log file. This required a few additions and updates to our code.

Using Output Buffer

We start by turning on the PHP output buffer (OB) and strting the output stream. This allows us to write verbose information from cURL to the output buffer. We must do this before starting the cURL handler.

ob_start();  
$out = fopen('php://output', 'w');

Then we needed to turn on verbose output information and define the output stream.

$curl = curl_init();
curl_setopt($curl, CURLOPT_VERBOSE, true);  
curl_setopt($curl, CURLOPT_STDERR, $out);  

After the curl_exec() call you can close the stream and fetch the information from the output buffer. To correctly capture all the information from cURL you must do it in this order - first close the stream and then get the contents.

fclose($out);  
$debug = ob_get_clean();

Now the $debug variable will hold a string containing data like this:

* Hostname was NOT found in DNS cache
*   Trying 23.21.53.220...
* Connected to requestb.in (23.21.53.220) port 80 (#0)
> POST /zn36z0zn HTTP/1.1
Host: requestb.in  
Accept: application/json  
Content-Length: 9  
Content-Type: application/x-www-form-urlencoded

* upload completely sent off: 9 out of 9 bytes
< HTTP/1.1 200 OK  
< Connection: keep-alive  
* Server gunicorn/19.3.0 is not blacklisted
< Server: gunicorn/19.3.0  
< Date: Fri, 18 Sep 2015 23:42:28 GMT  
< Content-Type: text/html; charset=utf-8  
< Content-Length: 2  
< Sponsored-By: https://www.runscope.com  
< Via: 1.1 vegur  
<  
* Connection #0 to host requestb.in left intact

Optionally, you can append the response body to the $debug variable. We did that to see what the service was returning when a unexpected error occurred.

We hope you found this article helpful. Please leave a comment below if you have any questions!

This is a follow-up to our post on Token Authentication Using Laravel + iOS. In our previous post we covered the backend API and how to configure Laravel token authentication. In this post we will build a hybrid iOS app using Apache Cordova along with the Onsen UI framework.

Our goal was to build a simple iOS to-do app that would connect to our backend API using token authentication (via JSON). This post will show you how to set up Cordova, Onsen UI and create the necessary templates for our to-do app. Let's get started.

Prerequisites

Before we start building the hybrid app we need to setup a base installation of Cordova + Onsen and configure our API server.

1. Install Cordova + Onsen

There is a fantastic guide on Onsen's site that walks you through this process. In summary you'll need to do the following:

1. Install NodeJS and NPM on your machine. We recommend using Brew for this by simply running brew install node and brew install npm.
2. Run the command sudo npm install -g cordova to install the Cordova package globally.
3. Download the master-detail template from the Onsen site.
4. Navigate to the unzipped folder and run the command cordova platform add ios to add the iOS SDK to the folder.
5. Ensure you have Xcode Command Line Tools by running xcode-select --install and ios-sim by running sudo npm install -g ios-sim.
6. You can now run cordova emulate to open the simulator and preview your app at any time.

2. Enable CORS

You need to enable CORS on your API server to allow remote access on the GET, POST and DELETE methods. There are different configuration requirements depending on the HTTP server your using.

Below are configurations for wide open CORS on Nginx and Apache. Use with caution and discretion. It's recommended that you understand what you are doing.

Nginx

Place this block of code inside your Nginx server block and then restart the Nginx service using service nginx restart on Ubuntu. You may optionally need to install the Nginx extras for support on the more_set_headers. You can do that by running this command on Ubuntu: apt-get install nginx-extras.

more_set_headers 'Access-Control-Allow-Origin: $http_origin';  
more_set_headers 'Access-Control-Allow-Methods: GET, POST, OPTIONS, PUT, DELETE, HEAD';  
more_set_headers 'Access-Control-Allow-Credentials: true';  
more_set_headers 'Access-Control-Allow-Headers: Origin,Content-Type,Accept,Authorization';

location / {  
    if ($request_method = 'OPTIONS') {
        more_set_headers 'Access-Control-Allow-Origin: $http_origin';
        more_set_headers 'Access-Control-Allow-Methods: GET, POST, OPTIONS, PUT, DELETE, HEAD';
        more_set_headers 'Access-Control-Max-Age: 1728000';
        more_set_headers 'Access-Control-Allow-Credentials: true';
        more_set_headers 'Access-Control-Allow-Headers: Origin,Content-Type,Accept,Authorization';
        more_set_headers 'Content-Type: text/plain; charset=UTF-8';
        more_set_headers 'Content-Length: 0';
        return 204;
    }
    try_files $uri $uri/ /index.php?$query_string;
}

Apache

On Apache it's a much easier. Simply add this line to your .htaccess file and restart Apache using service apache2 reload on Ubuntu.

Header set Access-Control-Allow-Origin "*"  

Views

Now that we have a base app up and running we need to create the following views for our app:

• Login
• Home
• Add Todo

Please note, all our views will live within the www/index.html file.

We want to clean this file up a little before starting. Lets remove the <style> block in the head and the entire <body> from the starter template. You'll be left with something like this:

<!DOCTYPE html>  
<!-- CSP support mode (required for Windows Universal apps): https://docs.angularjs.org/api/ng/directive/ngCsp -->  
<html lang="en" ng-app="app" ng-csp>  
<head>  
    <meta charset="utf-8" />
    <meta name="apple-mobile-web-app-capable" content="yes" />
    <meta name="mobile-web-app-capable" content="yes" />

    <!-- JS dependencies (order matters!) -->
    <script src="scripts/platformOverrides.js"></script>
    <script src="lib/angular/angular.js"></script>
    <script src="lib/onsen/js/onsenui.js"></script>

    <!-- CSS dependencies -->
    <link rel="stylesheet" href="lib/onsen/css/onsenui.css" />
    <link rel="stylesheet" href="lib/onsen/css/onsen-css-components-blue-basic-theme.css" />

    <!-- CSP support mode (required for Windows Universal apps) -->
    <link rel="stylesheet" href="lib/angular/angular-csp.css" />


    <!-- --------------- App init --------------- -->
    <script src="js/app.js"></script>

    <title>Onsen UI - Todo App</title>

</head>  
<body>  
    <!-- Cordova reference -->
    <script src="cordova.js"></script>
    <script src="scripts/index.js"></script>
    <!-- -->
</body>  
</html>  

Login View

The login view will contain the following elements:

• A basic form (email and password) with a submit button.
• A modal with a loading message: Please wait. Connecting....
• A button in the toolbar with a loading icon, which shows when we are fetching data from the API. We enable this using the function ng-show="isFetching".

Each view will eventually have its own Angular controller. For this view we have attached a LoginController which will execute at the beginning via the ng-init="init()" function. You will see later in this post how we are going to run this function.

The login view, in conjunction with the controller, will check if we have a token stored in local storage and verify if this token is still valid. If the token is valid it will pass the user to the Home template where the user will see their todos. If the token is invalid it will show the user a login form.

Place this code directly below the cordova-reference in the body.

<ons-navigator var="navi">  
    <ons-page ng-controller="LoginController" ng-init="init()">
        <ons-toolbar>
            <div class="center">Todo</div>
            <div class="right">
                <ons-toolbar-button ng-show="isFetching">
                    <ons-icon icon="ion-load-c" spin="true">
                </ons-toolbar-button>
            </div>
        </ons-toolbar>

        <div class="login-form" ng-hide="isLogged">

            <input type="email" class="text-input--underbar" ng-model="email" placeholder="Email" >
            <input type="password" class="text-input--underbar" ng-model="password" placeholder="Password">                
            <br><br>
            <ons-button modifier="large" class="login-button" ng-click="login()">Log In</ons-button>             

        </div>

        <ons-modal var="loginModal">
            <ons-icon icon="ion-loading-c" spin="true"></ons-icon>
            <br><br>
            Please wait.<br>Connecting...
        </ons-modal> 

    </ons-page>        
</ons-navigator>  

Home View

The home view will use the default Onsen UI list style to render the todos. In addition to listing the todo's we will include a few more UI features:

• A button to add todos.
• A loading icon that shows while todos are being added.
• A swipe to delete interaction. This is achieved by using the <ons-carousel> tag. This tag allows us to include 2 slides - one for the todo and another for the delete button.
• A modal to indicate when a todo is being deleted.

Add this block of code directly below your login view code.

<ons-template id="home.html">  
    <ons-page ng-controller="HomeController" ng-init="init()">
        <ons-toolbar>
            <div class="left">
                <ons-toolbar-button ng-show="isFetching">
                    <ons-icon icon="ion-load-c" spin="true">
                </ons-toolbar-button>
            </div>
            <div class="center">Todo List</div>
            <div class="right">
                <ons-toolbar-button ng-click="showAddTodo()">
                    <ons-icon icon="ion-compose">
                </ons-toolbar-button>
            </div>
        </ons-toolbar>

        <ons-list modifier="todo-list chevron">
            <ons-list-item class="item" ng-repeat="item in items">
                <ons-carousel style="height: 44px; width: 100%" swipeable initial-index="0" auto-scroll>
                    <ons-carousel-item>
                        <ons-button modifier="large--quiet" class="todo-btn">{{item.title}}</ons-button>
                    </ons-carousel-item>
                    <ons-carousel-item>
                        <ons-button modifier="large" class="delete-btn" ng-click="deleteTodo(item.id)">Delete</ons-button>
                    </ons-carousel-item>
                </ons-carousel>
            </ons-list-item>
        </ons-list>

        <ons-modal var="deleteModal">
            <ons-icon icon="ion-loading-c" spin="true"></ons-icon>
            <br><br>
            Deleting Todo...
        </ons-modal> 

    </ons-page>

</ons-template>

Add Todo View

This view contains a simple form and submit button for adding a new todo item. It will also contain a modal to let the user know when a todo is being saved to the API. This view will work with the addTodoController controller to create new todo items.

Add this block of code directly below your home view code.

<ons-template id="add.html">  
    <ons-page ng-controller="addTodoController">

        <ons-toolbar>
            <div class="left"><ons-back-button>Back</ons-back-button></div>
            <div class="center">Add Todo</div>
        </ons-toolbar>

        <ons-list modifier="inset" style="margin-top: 10px">
            <ons-list-item>
                <input type="text" class="text-input text-input--transparent" ng-model="todo" placeholder="Task" style="width: 100%">
            </ons-list-item>
        </ons-list>

        <div style="padding: 10px 9px">
            <ons-button modifier="large" style="margin: 0 auto;" ng-click="submitTodo()">Add Todo</ons-button>
        </div>

        <ons-modal var="todoModal">
            <ons-icon icon="ion-loading-c" spin="true"></ons-icon>
            <br><br>
            Adding Todo...
        </ons-modal> 

    </ons-page>
</ons-template>  

AngularJS Controllers

At this point you can run cordova emulate and you can preview the app. There isn't much to see, but if you did everything correctly up to this point you should see a simple screen with a "ToDo" title.

In order to bring the app to life we will need an AngularJS controller for each view. We will need to create the following controllers:

• LoginController
• HomeController
• addTodoController

All controller related logic will go in the www/js/app.js file. Before starting, let's strip some of the default code so we end up with the following:

(function(){
  'use strict';
})();

Next, we need to define some variables that will help simplify the process for us. The api variable will be our API endpoint, and the Object.toparams function will convert parameters to JSON in our POST methods. Add this within the main function, after the use strict declaration. Be sure to update the api variable.

  var api = 'http://yourserver.com/api/';
  var module = angular.module('app', ['onsen']);

  Object.toparams = function ObjecttoParams(obj) {
    var p = [];
    for (var key in obj) {
      p.push(key + '=' + encodeURIComponent(obj[key]));
      console.log(obj[key]);
    }
    return p.join('&');
  };

LoginController

Our login controller has two main functions: init() and login().

The init() funtion will run on load and check if we have a key called token in local storage. If the token exists the controller will attempt to validate it by sending a GET request to the validate_token route. If the token is valid, it will redirect the user to the home view. If not, it show the login form.

The login() function will pass the email and password input values as a POST method to the login route. If the credentials are valid, the API will return a token which is stored in the local storage. The user will then be redirected to the home view. If the credentials are invalid, the controller will return an error and show the login form again.

Place this block of code below your ObjecttoParams function.

module.controller('LoginController', function($scope, $http) {

  $scope.isLogged = true;
  $scope.isFetching = false;
  $scope.email = '';
  $scope.password = '';

  $scope.init = function(){
    $scope.isFetching = true;

    if (window.localStorage.getItem("token") !== null ) {
      $http.get(api+'validate_token?token='+window.localStorage.getItem("token")).success(
        function(response) {
          console.log(response);
          if( typeof(response.status) !== "undefined"){
            $scope.isLogged = true;
            $scope.isFetching = false;
            $scope.navi.pushPage('home.html');
          } else {
            $scope.isLogged = false;
            $scope.isFetching = false;
          }
        }
      ).error(function(response){
        $scope.isFetching = false;
      });

    } else {    
      $scope.isLogged = false;
      $scope.isFetching = false;
    } 
  };

  $scope.login = function(){
    if($scope.email==='' || $scope.password===''){     
      ons.notification.alert({message: "You can't leave any fields empty"});
    } else {   
      loginModal.show();
      var data = {
        email: $scope.email,
        password: $scope.password
      };

      $http({
        url: api+'login',
        method: 'POST',
        data: Object.toparams(data),
        headers: {'Content-Type': 'application/x-www-form-urlencoded'}
      }).success(function (response) {
        console.log(response);
        if( typeof(response.token) !== "undefined"){
          window.localStorage.setItem("token", response.token);
          loginModal.hide();
          $scope.navi.pushPage('home.html');
        } else {
          loginModal.hide();
          ons.notification.alert({message: 'Your username/password was incorrect, please try again.'});
        }
        }).error(function(response){
          loginModal.hide();
          $scope.isFetching = false;
      });
    }
  };
});

HomeController

The home controller has 3 main functions: init(), showAddTodo() and deleteTodo().

The init() function will run on load and send a GET request to the todo route (with the token from our local storage). The API will validate the token and respond with a JSON object containing all the todo items for the user. We will assign this list of todos to the $scope.items variable, and include it on the ng-repeat in the home view.

The showAddTodo() function redirects users to the add todo view.

The deleteTodo(), when triggered, will send a DELETE request to the selected Todo upon a successful response it will send you back to the home.html page and also restart the init() function so the app can grab the fresh list of Todos from the API.

We also included an event called rootScope:refreshtodos we are gonna make use of this one in the next controller, when I trigger this event in the next controller I will just refresh the list of Todos by running the init() function again.

Place this block of code below our LoginController.

module.controller('HomeController', function($scope, $http, $rootScope) {  
  $scope.isFetching = true;
  $scope.items = [];

  $scope.init = function(){

    $http.get(api+'todo?token='+window.localStorage.getItem("token")).success(
      function(response) {
        console.log(response);
        if( typeof(response.status) !== "undefined"){

          $scope.isFetching = false;
          $scope.items = response.todos;

        } else {
          ons.notification.alert({message: 'There was an error connecting to the API.'});
          $scope.isFetching = false;
        }
      }
    ).error(function(response){
      ons.notification.alert({message: 'There was an error connecting to the API.'});
      $scope.isFetching = false;
    });

  };

  $scope.showAddTodo = function(index) {
    $scope.navi.pushPage('add.html');
  };

  $scope.deleteTodo = function(id){
    deleteModal.show();

    $http({
      url: api+'todo/'+id+'?token='+window.localStorage.getItem("token"),
      method: 'DELETE',
      headers: {'Content-Type': 'application/x-www-form-urlencoded'}
    }).success(function (response) {
      console.log(response);
      if( typeof(response.status) !== "undefined"){
        deleteModal.hide();
        $scope.init();
      } else {
        deleteModal.hide();
        ons.notification.alert({message: 'There was an error connecting to the API, please try again.'});
      }
    });
  };

  $rootScope.$on('rootScope:refreshtodos', function (event, data) {
    $scope.init();
  });

});

addTodoController

This controller has one function: submitTodo() which will send a POST request to the todo route. If we receive a successful response, the controller will broadcast the event to rootScope:refreshtodos so the home view can refresh the todo list. It will then redirect the user back to the home view.

Place this block of code below your HomeController.

module.controller('addTodoController', function($scope, $http, $rootScope) {

  $scope.isFetching = false;
  $scope.todo = '';

  $scope.submitTodo = function(){
    if($scope.todo===''){    
      ons.notification.alert({message: "You can't leave the field empty"});
    } else {

      todoModal.show();
      var data = {
        title: $scope.todo
      };

      $http({
        url: api+'todo?token='+window.localStorage.getItem("token"),
        method: 'POST',
        data: Object.toparams(data),
        headers: {'Content-Type': 'application/x-www-form-urlencoded'}
      }).success(function (response) {
        console.log(response);
        if( typeof(response.status) !== "undefined"){
          todoModal.hide();
          $rootScope.$broadcast('rootScope:refreshtodos', 'RefreshTodos');
          $scope.navi.popPage();
        } else {
          todoModal.hide();
          ons.notification.alert({message: 'There was an error connecting to the API, please try again.'});
        }
      });
    }
  };
});

Conclusion

You may now test the app by running cordova emulate from the root project folder. At this point feel free to customize the app styles or add additional features.

Distribution

Once everything is running you can prepare the app for distribution using three simple steps:

1. Run the command cordova prepare to prepare the Xcode files.
2. Open the platforms/ios/Onsen UI Project.xcodeproj file in Xcode.
3. Generate an .ipa file by navigating to Project -> Archive.

Repository

You can download this project directly from our Github repo Hybrid Onsen App.

Thanks for reading! If you have any comments or questions please leave them below.

Just recently we were tasked with developing a web + iOS app for a client. We wanted to leverage the speed & simplicity of building for the web using Laravel, but utilize this same backend architecture for the iOS app. This meant we needed to create a RESTful API to interact with our iOS app.

As with most apps, this one required authentication. We decided to use tokens for our authentication module. This article is apart of a two part series designed to demonstrate how to build a basic API which will eventually interact with an iOS to-do app. It will then demonstrate how to connect the iOS app & API using a token authentication system (via JSON).

This particular article will cover the API that runs behind the scenes. We will be posting a follow up article in the coming weeks that covers the actual iOS development.

Our goals for this app are to create an API which will allow us to:

• Login
• Verify a token
• Get / Add / Delete to-do items

The tools we will use:

• Laravel - version 4.2
• Dingo - RESTful API package
• JWT Auth - JSON web token auth (works well with Dingo)
  
  

As always, we encourage you to check out the code on our Github. It may be helpful to follow along this way.

Basic Framework

I'll assume that you already have a standard installation of Laravel 4.2 up and running. If you need to do that first, their site provides a great quickstart guide.

Configure Dingo & JWT Auth

Add both these packages to your composer.json file:

"dingo/api": "0.8.*",
"tymon/jwt-auth": "0.4.*"

Then run composer update to grab the repository files.

dingo/api

We strongly recommend that you read the wiki for this plugin. Here is a quick summary of how to get things up and running:

1. Configure the Provider & Alias inside app/config/app.php
2. Publish the package configuration file by running php artisan config:publish dingo/api
3. Set the prefix variable to api.
4. Add JSON web tokens as an authentication provider inside your app/config/packages/dingo/config.php file. The auth array should then look something like this:

'auth' => [  
    'basic' => function ($app) {
        return new Dingo\Api\Auth\BasicProvider($app['auth']);
    },
    'jwt' => function ($app) {
        return new Dingo\Api\Auth\JWTProvider($app['tymon.jwt.auth']);
    }
],

jwt-auth

The developer for this plugin has a nice getting started guide that walks you through installation.

Database Design

Our goal is to build a to-do list module with authentication. Let's create the necessary tables.

lists

This table contains the master lists for our users. We can identify the list owner using the user_id column.

todos

This table contains all the child list items (also know as to-do items). We can identify the parent list using the list_id column.

users

This table will save all the users login information.

We can create all three tables by writing and running a migration. Run the artsian command php artisan migrate:make todo_data_structure

Then add the following inside the newly created TodoDataStructure class:

public function up() {

    Schema::create('lists', function(Blueprint $table) {
        $table->increments('id');
        $table->integer('user_id');
        $table->string('title');
        $table->timestamps();
    });

    Schema::create('todos', function(Blueprint $table) {
        $table->increments('id');
        $table->integer('list_id');            
        $table->string('title');
        $table->timestamps();
    });

    Schema::create('users', function(Blueprint $table) {
        $table->increments('id');
        $table->string('email')->unique();
        $table->string('password');
        $table->timestamps();
    });     

}

public function down() {  
    Schema::drop('lists');
    Schema::drop('todos');
    Schema::drop('users');
}

Setup Models

We will setup 3 models to interact with the API routes and database: User, Todo, TodoList. Add the following to your existing models/User.php file:

public function lists() {  
    return $this->hasMany('TodoList', 'user_id');
}

Create a new file models/Todo.php and add the following:

<?php

class Todo extends Eloquent {

  protected $fillable = ['list_id', 'title'];
  protected $table = 'todos';

  // Get the list of this todo
  public function todoList(){
    return $this->belongsTo('TodoList', 'list_id');
  }

}

Create a new file models/TodoList.php and add the following:

<?php

class TodoList extends Eloquent {

  protected $fillable = ['user_id', 'title'];
  protected $table = 'lists';

  // Get the todos from this list
  public function todos(){
    return $this->hasMany('Todo', 'list_id');
  }

}

Configure API Routes

At this point you now have a base skeleton for the application up and running. Now its time to code the API routes. We will create an api array inside the routes file. This will allow us to protect a route with an auth token by simply adding the 'protected' => true parameter. The api array will look like this:

Route::api('v1', function () {  
});

Login & Validate Token

In order to properly use the token authentication, we will need to code a couple of routes: login and validate_token. The login route will give us a valid token if our user and password is correct. Otherwise, it will just throw an error. Place these routes inside your api array:

Route::post('login', [  
    'uses'      => 'AuthenticateController@authenticate',
    'as'        => 'api.login'
]);

The validate_token route will check that the token we saved is still valid. We will verify the token everytime the app is opened:

Route::get('validate_token',  array(  
    'protected' => true,
    'as'        =>  'api.validate_token',
    'uses'        => 'AuthenticateController@validateToken'
));

Todo Routes

And finally a the routes for viewing, adding and deleting Todos from each user's list:

Route::get('todo', [  
    'uses'      => 'TodoController@index',
    'as'        => 'todo.index',
    'protected' => true
]);

Route::post('todo', [  
    'uses'      => 'TodoController@store',
    'as'        => 'todo.store',
    'protected' => true
]);

Route::delete('todo/{id}', [  
    'uses'      => 'TodoController@destroy',
    'as'        => 'todo.destroy',
    'protected' => true
]);                

Controllers

Let's code the methods for our controllers: AuthController and TodoController.

AuthController

This particular controller will need two methods. First, the authenticate method will verify the user email and password. If the credentials are valid it will generate a token and return it as JSON. If the credentials are not valid it will return a 401 error with the message "invalid_credentials".

Second, the validate_token method is used to verify the validity of the token. If the token is still valid it will return a success message. If the token is not valid, it will return an error exception based on the JWT errors.

To accomplish this we will create a new file controllers/AuthenticateController.php and put the following code in it:

<?php  
use Tymon\JWTAuth\Exceptions\JWTException;

class AuthenticateController extends \BaseController{

  public function authenticate() {

    // Get credentials from the request
    $credentials = Input::only('email', 'password');

    try {
      // Attempt to verify the credentials and create a token for the user.
      if (! $token = JWTAuth::attempt($credentials)) {
        return API::response()->array(['error' => 'invalid_credentials'])->statusCode(401);
      }
    } catch (JWTException $e) {
      // Something went wrong - let the app know.
      return API::response()->array(['error' => 'could_not_create_token'])->statusCode(500);
    }

    // Return success.
    return compact('token');
  }

  public function validateToken(){
    return API::response()->array(['status' => 'success'])->statusCode(200);
  }

}
?>         

TodoController

This particular controller will need a method for each todo action: add and delete.

Create a new file controllers/TodoController.php and put the following code in it:

<?php

class TodoController extends \BaseController{

  public function index(){

    $user = API::user();
    $userList = TodoList::find($user->lists()->first()->id);

    if($userList){

      $todos = $userList->todos()->get();

      return API::response()->array(['status' => 'success', 'todos' => $todos ]);

    }

    return API::response()->array(['status' => 'failed', 'message' => 'There was an error. Please try again.' ])->statusCode(401);

  }

  public function store(){

    $validation = Validator::make(Input::all(), [
      'title' => 'required'
    ] );

    if($validation->fails()){
      return API::response()->array(['status' => 'failed', 'message' => $validation])->statusCode(200);
    }

    $user = API::user();

    $userList = TodoList::find($user->lists()->first()->id);

    try{

      $todo = new Todo([
        'title'    => Input::get('title'),
      ]);

      $todo->save();

      if($todo && $userList){
        $userList->todos()->save($todo);
      }

      return API::response()->array(['status' => 'success', 'message' => 'Todo was added!']);

    } catch(\Exception $e){

        return API::response()->array(['status' => 'failed', 'message' => 'There was an error. Please try again.' ])->statusCode(401);

    }

  }

  public function destroy($id){

    $user = API::user();
    $userList = TodoList::find($user->lists()->first()->id);

    $todo = Todo::find($id);

    if($todo && $todo->list_id == $userList->id){

      $todo->delete();

      return API::response()->array(['status' => 'success', 'message' => "Todo was deleted." ])->statusCode(200);

    }

    return API::response()->array(['status' => 'failed', 'message' => 'There was an error. Please try again.' ])->statusCode(401);

  }

}

?>                

Testing

Now that we have all the code in place, lets make sure its working correctly.

Manually Create User & Todo List

First you will need to add a user to the database. Run the following SQL command, or manually insert a user into the database:

INSERT INTO `users` (`id`, `email`, `password`, `created_at`, `updated_at`)  
VALUES (1, 'hello@kettle.io', '[password]', '2015-06-16 02:32:51', '2015-06-16 02:32:51');  

Note that we will need to generate a hashed password in place of [password]. Since this is dependent on your installations app key you can generate a password through the artisan tinker command.

1. Move to your project directory and run the php artisan tinker command
2. Generate a password using the following command echo Hash::make('abc123');
3. Copy the output and use it in the password column of your database.

Next, we need to add a new list to the database and associate it with our user. Once we've built the iOS app these lists will be dynamically generated in the database. However, for the purposes of testing we will need to manually add a list. Run the following SQL command, or manually insert a user into the database (assuming your newly created user has the ID 1):

INSERT INTO `lists` (`id`, `user_id`, `title`, `created_at`, `updated_at`)  
VALUES (1, 1, 'Test List', '2015-06-16 02:32:51', '2015-06-16 02:32:51');  

Generate Token

Now that you have created a user and todo list lets go ahead and ensure that the app is creating and validating tokens correctly. For this I prefer to use a Chrome extension called Postman. You'll need to send a POST request to the /api/login route including the email and password. If you recieve a token back, everything is working as expected. You'll use this token to authenticate subsequent calls to the API. It should look something like this in Postman:

Create Todo Item

Send a POST request to /api/todo with your token as a query string and the desired todo title as a form input. This will create the todo item in your database and map it to the specified user. For simplicity sake, each user only has one list. It will look like this:

View Todos

Send a GET request to /api/todo with your token as a query string. This will return a JSON object with all the todo items for that particular user. It will look like this:

Delete Todo Item

Send a DELETE request to /api/todo/{id}?token=yourtoken - where ID is the ID of the todo item you wish to delete. It should look something like this:

Conclusion

Thanks for reading. Please visit again for our follow up article on the iOS portion of this project.

If you have any comments or questions please leave them below. Remember you can also report issues and download this project directly from our Github.

There has been a lot of buzz around Go (golang) lately. I don't tend to be an earlier adopter for new languages since a strong community and proven platform are critical when building software intended for production enviroments.

Go has matured quite a bit since it was introduced in 2007, so I decided to take a deeper look this week. My research concluded that it makes a good fit for scalabale (enterprise level) web apps, and API's in specific.

Go supports concurrency and multi-threading on the language level, right out of the box. It also enforces strict syntax and formatting rules - making your code more standardized and less error prone. There are other advantages like built in testing tools and fast compiling speed, but I won't harp on those too much. I don't consider either of those a differentiating factor.

I wanted to give Go a try and build a practical application that we would normally develop here at Kettle. A simple API, hooked up to MySQL seemed like the perfect concept.

Lets do this!

Prerequisites

Since I wanted to test the full life cycle of a Go application I needed to setup a remote enviroment on Heroku. I won't go into too much detail on how to set these up, but below are some resources that I found extremely helpful:

• Setting up Go locally on a Mac isn't very straight forward. You can use Homebrew here, but I'd suggest referencing the Offical Site & Ryan Day's Guide
• Setting up Go with Heroku was pretty straight forward using Mark McGranaghan's guide

The Code

I found a nifty framework called Martini that was designed to make the code more modular and quicker to write. I came up with the following code for my super simple sudo-API:

package main

import (  
    "github.com/go-martini/martini"
    _ "github.com/go-sql-driver/mysql"
    "database/sql"
    "fmt"
)

func main() {

    // Connect to database
    db, err := sql.Open("mysql",
        "user:pass@tcp(localhost:3306)/tableName")


    // Fire up martini & define route
    m := martini.Classic()
    m.Get("/api/:id", func(params martini.Params) string {   

          // Define variables
          var str string
          var userId =  params["id"]

          // Run MySQL query against param
          err = db.QueryRow(
              "select name from users where id = ?", userId).Scan(&str)


          // Check for errors and log if they exist     
          if err != nil && err != sql.ErrNoRows {
              fmt.Println(err)
          } 

          return str 

    })

  m.Run()
  defer db.Close()

}

Just How Fast Is It?

Now that I had a functioning piece of Go code I wanted to see just how fast the language was. After hearing all the praises of the language I needed to see first hand how good/bad it was.

To test the app I used a service called Loader.io (awesome tool by the way). I had Loader query the service with 4,000 clients (making 2 requests each) over the course of 60 seconds and these are the results I saw:

The results are astounding. I was running a free Heroku instance and free database from ClearDB. Regardless of those limitations, the app managed an average response of 35ms with zero errors. I found this impressive for three reasons:

• The response times were quick and consistent
• The error rate was zero which is important
• The app handled many requests per second on limited server resources without issue (think: cheap infastructure costs at scale)

Compared to Laravel 5

My intention here was to learn Go and see if it was really as good as people said. After seeing the results, I'm pretty convinced it is good. However, I was curious to see how it would stack up against Laravel 5, an awesome PHP framework we use quite often here at Kettle. I knew this wasn't a fair comparision, but I did it anyway. Here is the code we used in Laravel:

routes.php

<?php

Route::get('/', 'WelcomeController@index');  
Route::get('/api/{user}', 'ApiController@user');  

ApiController.php

<?php 

namespace App\Http\Controllers;  
use DB;

class ApiController extends Controller {

    public function user($user) {
        $response = DB::table('users')->where('id', $user)->first();

        return $response->name;
    }

}

I ran the same test again - 4,000 clients over the course of 60 seconds, each making 2 requests. These are the results observed:

Conclusion

As expected, Laravel performed much slower than Go. I believe Go is a great choice for something like an enterprise API. It's quick and predicable. This makes it easy to scale.

Laravel has it's place too - it allows us to build robust web apps, quickly. That makes it a great fit for projects that require quick turn around and agile development.

We hope you enjoyed this article. Please feel free to leave comments below.

One part of testing any application is seeing how it reacts to a volume of data. When building an application with Laravel, we use a great package called Faker to make this process quick. Their API allows us to painlessly add a ton of content to our application.

This is an add-on to our previous post Dynamic Pagination using Laravel & AngularJS and will show you how to add 50 "fake" posts to your application for testing.

The tools we will use:

• Laravel 4.2
• Faker

Let's Get Started

I will assume that you already have a database setup. I will also assume you've created a posts table with the following structure:

• id
• title
• content

Install Faker

Our fist step will be to install Faker within our Laravel application. You'll need to use composer to install the package. Either through terminal or the composer.json file.

Using Terminal

1. cd /Directory-of-Laravel-App
2. sudo composer require fzaninotto/faker --dev

Using composer.json

1. Open up your composer.json file.
2. Add the following snippet to your require-dev array: "fzaninotto/faker": "1.3.*@dev"
3. cd /Directory-of-Laravel-App
4. Run the command composer update

Create Seeder File & Define Data

The next step is to create a file which will generate the fake data for us. Since we are trying to seed the posts table, lets go ahead and create a file called PostsTableSeeder.php inside app/database/seeds/. Faker adds on to the existing seeder class that ships with Laravel.

The best part of Faker is the flexability of its API. We can easily insert sentences, paragraphs, images, dates, names, phone numbers and more. A complete list of content types is available here.

As you'll see we opted to use the sentence (post title) and paragraph (post description) content types.

<?php

use Faker\Factory as Faker;

class PostsTableSeeder extends Seeder {

    public function run() {
        $faker = Faker::create();

        foreach(range(1, 50) as $index) {
            Post::create([
                'title'     => $faker->sentence(5),
                'content'   => $faker->paragraph(4)
            ]);
        }
    }
}

Add to Seeder Class

Our last step (I told you Faker made this quick), is to add the Posts seeder we just created to our master seeder class. Go ahead and edit the file DatabaseSeeder.php within app/database/seeds/.

Assuming you haven't made any changes to the existing class, your file should look like this:

<?php

use Illuminate\Database\Seeder;  
use Illuminate\Database\Eloquent\Model;

class DatabaseSeeder extends Seeder {

    public function run() {
        Model::unguard();
        DB::table('posts')->truncate(); //If you want to reset to 50. See note below.
         $this->call('PostsTableSeeder');
    }
}

Now everytime you run php artisan db:seed in your terminal you will be adding 50 more posts to your database.

Note: If you don't want to keep adding 50 additional posts everytime you run db:seed you will want to include this line of code. It will remove the previous database entries before inserting the new dummy content.

Obviously we are using this library within a test enviroment. So when you move the application to production you will most certainly want to remove the truncate(); function from your code. Otherwise you may accidentaly run db:seed and erase data from your production database.

Today we will be combining the Laravel framework with Google's Angular JS. This has a variety of uses, but our article will show you how to setup posts within a blog. We will be implementing dynamic pagination using the built in Laravel paginate option.

At the time of this post we were unable to find existing documentation on this approach. We are making this post in hopes that you find it helpful. The entire code can be found on our GitHub.

Tools
For this article we will be using the following tools. It's important that you are familar with the frameworks in order to use the system to it's full advantages.

• AngularJS JS framework (we used 1.3.0)
• Laravel PHP framework (we used 4.2)
• Twitter Bootstrap - HTML/CSS framework

Let's Get Started

I'll assume that you already have a standard installation of Laravel up and running. If you need to do that first, their site provides a great quickstart guide.

Create Routes & Views

First we will create the routes and view needed to show your blogs posts. We'll build something simple using the grid styles provided by Bootstrap. You can customize this later once the system is hooked up. In this example we'll include 5 posts and next/back buttons.

Inside the routes file we will add a route for the JSON data and one for the actual posts page. Add the following to your app/routes.php file.

Route::get('posts', function() {  
    return Response::view('posts');
});

Route::get('posts-json', array(  
  'as' => 'posts-json', 
  'uses' => 'PostController@json' 
));

Create a new file called posts.blade.php in /app/views and place the following code in it. Note that we have included the Bootstrap library.

<!DOCTYPE html>  
<html lang="en">  
  <head>
    <link href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.2/css/bootstrap.min.css" rel="stylesheet">
    <title>My Blog</title>
  </head>
  <body>  
    <div class="col-md-6 col-md-offset-3">  
      <h2>Posts</h2>

    </div> 
  </body>  
</html>  

Create Database Table & Controller

We will create a new model for our posts. We will name the new model Post. We will also create a new table to save all my posts with some basic structure. We will call this table posts and give it columns for:

• ID (auto-increment)
• Title (varchar, 100)
• Content (varchar, 255)

You should really use php artisan migrate:make create_posts_table here but I won't go into detail on that. Hopefully your familar with the built in migrations.

Now that we have the posts table setup we are going to populate it with some data. For testing I like to use the Faker library. If you are unfamiliar with how to use Faker I suggest taking a look at our article on how to seed a database with dummy data in Laravel.

Now that we have the database table created and populated, lets create a controller for the posts. We will use this to paginate through the posts and fetch their data. We will create a new file in the app/controllers folder and call it PostController.php. Inside we will create a new function json that will geneate the JSON data:

<?php  
class PostController extends BaseController {

    public function json(){  
        $posts = Post::paginate(5); // 5 is the number of items to show per page
        return Response::json($posts);
    }

}

Create "post" Model

We will now create a post model to connect our new controller to the post table. Create a new file called Post.php in the app/models directory. Inside the new file add the following code.

<?php

class Post extends Eloquent  {  
    protected $fillable = ['title', 'content'];
    protected $table = 'posts';
    protected $hidden = array();
}

Add & Configure AngularJS

It is time to add AngularJS to the page and unlock all the potential that this framework can offer. Go ahead and include the library in your page. You will also need to create a JS file to place your code.

<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.3.2/angular.min.js"></script>  
<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.3.2/angular-resource.min.js"></script>  
<script src="js/app.js"></script>  

Important!
Laravel's blade templates use the same opening/closing brackets as AngularJS {{ }}. We must configure AngularJS to use [[ ]] instead to avoid conflicts. We can do that by adding the following code to our newly created js/app.js file.

var app = angular.module('kettle', ['ngResource'],  
 function($interpolateProvider) {
    $interpolateProvider.startSymbol('[[');
    $interpolateProvider.endSymbol(']]');
});

Configure View File

Now that we have enabled AngularJS lets modify the HTML view to render it. Please note that we have now included ng-app in the body tag. This is critical.

  <body ng-app="kettle">  
    <div class="col-md-6 col-md-offset-3" ng-controller="postController" ng-init="getPosts()">  

      <h2>Posts</h2>

      <div class="list-group" >  
        <a href="#" class="list-group-item" ng-repeat="post in posts" >
          <h4 class="list-group-item-heading">[[ post.title ]]</h4>
          <p class="list-group-item-text">[[ post.content ]]</p>
        </a>
      </div>

      <div>
        <posts-pagination></posts-pagination>
      </div>

    </div>  
  </body>  

You will notice that we have used the directive in place of buttons. This will allow us to define the pagination based on the bootstrap styles inside our js/app.js file.

app.directive('postsPagination', function(){  
   return{
      restrict: 'E',
      template: '<ul class="pagination">'+
        '<li ng-show="currentPage != 1"><a href="javascript:void(0)" ng-click="getPosts(1)">&laquo;</a></li>'+
        '<li ng-show="currentPage != 1"><a href="javascript:void(0)" ng-click="getPosts(currentPage-1)">&lsaquo; Prev</a></li>'+
        '<li ng-repeat="i in range" ng-class="{active : currentPage == i}">'+
            '<a href="javascript:void(0)" ng-click="getPosts(i)">{{i}}</a>'+
        '</li>'+
        '<li ng-show="currentPage != totalPages"><a href="javascript:void(0)" ng-click="getPosts(currentPage+1)">Next &rsaquo;</a></li>'+
        '<li ng-show="currentPage != totalPages"><a href="javascript:void(0)" ng-click="getPosts(totalPages)">&raquo;</a></li>'+
      '</ul>'
   };
});

Fetch & Parse JSON Using AngularJS

If you remember in the beginning we created the route /posts-json/ to serve up the post information using JSON. Now its time to use it. When you load that page you should see something like this.

The data key in the JSON is the one that holds the content of all our posts. When we paginate with Laravel all we need to do is add a page parameter to the URL and Laravel will handle the rest. For example, /posts-json?page=2 would yield the second page of results.

Now its time to write the code snippet for AngularJS that will fetch and parse the JSON + enable the pagination functions. This requires that we write a new controller called postController in our app.js file.

Update! We've updated the code below to use pagination with ngResource. We'll leave the old code commented inline, in case you want to continue using that!

app.factory( 'Post', function($resource){  
  return $resource('post');
});

app.controller('postController', [ '$http', '$scope', function($http, $scope){

  $scope.posts = [];
  $scope.totalPages = 0;
  $scope.currentPage = 1;
  $scope.range = [];

  $scope.getPosts = function(pageNumber){

    if(pageNumber===undefined){
      pageNumber = '1';
    }
    Post.get({page: pageNumber},function(response){
    // Old pagination style using http
    // $http.get('/posts-json?page='+pageNumber).success(function(response) { 

      $scope.posts        = response.data;
      $scope.totalPages   = response.last_page;
      $scope.currentPage  = response.current_page;

      // Pagination Range
      var pages = [];

      for(var i=1;i<=response.last_page;i++) {          
        pages.push(i);
      }

      $scope.range = pages; 

    });

  };

}]);

Once you've added this code you should see the final result by visiting the /posts you created in the beginning. If everything went smooth your final result should look something like this:

Summary

We hope this article was helpful to you. If you have any comments please leave them below - thanks!

Today we are launching our blog. This marks a new addition to our business and is something we plan to contribute to on a regular basis moving forward in 2015.

You can expect a new post every 3-4 weeks. I expect the blog to consist of a few different things for the forseeable future:

1. Tutorials walking you through a development issue we faced this month.
2. How we made business or dev-ops related decisions.
3. Kettle projects and Kettle client news!

I hope to see our blog expand and evolve beyond this...with the help and guidance of our readers of course!

Until next time.