Using UUIDs with Eloquent in Laravel

Note: this article I wrote originally appeared on the Humaan blog.

snowflake-illustration

Recently there have been some interesting discussions and posts about the use of incrementing IDs in websites and whether that’s accidental information leakage or not. An excellent article by Phil Sturgeon showed that you can learn a lot about how much a service is being used with just a few minutes of investigation. An incrementing identifier (or ID) is an integer that starts at 1, and is increased by 1 each time a new record is saved to the database. A UUID (or universally-unique identifier), on the other hand, is a 36 character long identifier made up of 32 alphanumeric characters with four hyphens in amongst it. Due to the length of a UUID it is much more difficult to guess UUIDs, let alone figure out how many users have registered on a site.

Note! The code in this article was written for Laravel 5.1 LTS

In the last few months we started working on a new project in the Humaan labs and the discussion of IDs versus UUIDs came up. This particular project was to be written in Laravel, and while Laravel uses auto-incrementing IDs as the unique key for a model, it’s easy enough to switch those out and use UUIDs instead. Having said that, we decided that we wanted to keep the auto-incrementing IDs for internal use but use UUIDs for user-accessible records (and by that I mean records where a URI would need to be generated like /post/uuid-here, for example).

So, I was tasked with implementing UUIDs for models in Laravel which, in itself, isn’t a difficult task. There were a few caveats however:

  • Not all models will have UUIDs so it can’t be on the base Model class
  • It must be reusable so adding or removing it from a Model doesn’t involve copying/removing a chunk of code each time
  • Users must never be able to generate their own UUIDs nor can they override them
  • However I implement UUIDs there must be a method that allows me to supply an auto-incrementing ID, or a UUID, which will return the requested record

Given that it must be reusable and can’t be on the base Model class in Laravel (and by that I mean the one in app/models, not the actual Eloquent class itself), I decided to go with a trait (truth be told I actually initially went with an abstract class but having to override the parent boot method just didn’t seem right). While there’s no defined place to store traits in Laravel I created a folder in the app/ directory called Traits. The name of our trait is UuidModel so, naturally, the file name is UuidModel.php.

When working with UUIDs in PHP there’s no point trying to roll your own solution to create and validate UUIDs. Time and time again developers have been caught out implementing their own half-baked solutions (hello rot13 or base64 as “hashing” or “encryption“), so we’re going to use a well-known PHP library that will generate the UUIDs for us. Open up your terminal and enter in composer require ramsey/uuid and press enter. Alternatively you can put "ramsey/uuid": "^2.8" in your composer.json file and run composer install. Once that library has installed, we’re ready to go!

Pretty basic so far – just a namespace, use statement, and the trait itself. If you look at the source for the base Eloquent Model class (for reference: Illuminate\Database\Eloquent\Model) you’ll see there’s a method pretty early on in the piece called bootTraits. This is called by the booter and looks for any traits on the model that have a static method starting in boot. This means that rather than having to do a nasty like this:

protected static function boot()
{
      parent::boot();

      // our stuff here
}

And perhaps forgetting to call parent::boot(); in a sub class (and thus causing Eloquent to implode) we can safely separate out our boot logic without interfering with the rest of the boot process. On a side note, if there’s one BIG tip I can give anyone working with Laravel it’s to check out the source code. You’ll find so many handy methods that will make your life much easier (and quicker too). Anyway, now that we’ve figured out we can add to the booting process, we’re going to make our booter and then attach some event listeners to the model so we can watch for specific events.

The boot method for the trait must be the same as the trait name with boot prepended. So, UuidModel becomes bootUuidModel, PickleCat becomes bootPickleCat, you get the idea. Check out the method class_basename in the Illuminate\Support\helpers.php file to see what it looks for. As mentioned before, the method must be static so don’t forget that keyword otherwise your booter will never run.

First off, we want to ensure that users can’t be sneaky and assign their own UUIDs (this is assuming your FormRequests, controllers, models, and what not prevent user input from setting the UUID) we’ll need to watch the creating event. This alone is perfect for preventing user input from wiping out your UUID but also saves you having to add another line to each instance where you create a model with UUIDs and generate it. What it doesn’t count for, however, is updates to the model. Thankfully, there’s an event for saving updates to a model so we can bind to that too. Let’s get started, shall we?

As mentioned in the Laravel Eloquent documentation, you can use a Service Provider to listen in on Eloquent events, or you can just use the late static binding static keyword to reference the current class in runtime! Simply put, the line static::creating will be called when that event is fired on the current model.

We’ll use a anonymous function as the creating event callback and supply the current model as a parameter with the name $model. Assign the attribute uuid with our UUID that we generate by calling the uuid4() method, and getting its value as a string (without the toString() method, the UUID is actually an array with 6 values). Pretty simple really. The creating event fires just before the model is persisted to the database, so unless you have a trait that boots after, your assigned UUID won’t be overwritten. If you’re curious on where this event is in the workflow check out the method performInsert on the aforementioned Eloquent model class.

Once again in this boot method we’ll bind to event but this time we’ll bind to the saving event. This one is a little more complete, mainly because we have to verify that the UUID hasn’t been altered in some way (whether that be maliciously, or accidental – although that’d never happen, we both know programmers never make mistakes). Whenever you’re interacting with an Eloquent model, you’ll notice that there’s an attributes property along with an original property. If you were to make a modification to the Eloquent model during the application lifecycle the changed value is stored in the attributes property. Using this information, we can safely check to see if the UUID stored in attributes is identical to the UUID in original.

Using the handy Eloquent method getOriginal, we can obtain the original UUID value and compare it. If the UUID has been changed somehow, revert it back. If the UUID hasn’t changed, we do nothing. This alone will give you UUIDs for a model in Laravel, but what if you want a reliable way to search for a particular model without having to write Query Builder statements every time? Look no further, I have the solution below.

As you may recall, Eloquent models have the fantastic ability that allow you to generate custom Query Builder queries on the model that can be reused time and time again. If you’re not familiar with these “scope” queries take a look at the “Query Scopes” section in the Eloquent documentation. I’ve defined two queries that we can use – one called scopeUuid, which receives and searches for a UUID only, and another query called scopeIdOrUuid, which can take either an ID or a UUID, and find the associated record. In both my scoped queries, I’m going to add an extra parameter so I can either return the model directly or return the Query Builder instance. This is more of a convenience thing rather than anything else – I find I’m usually just fetching a model straight rather than using the Query Builder, but adding it in means I can have the best of both worlds.

On to scopeUuid:

As with our magic booter method, all methods starting with scope are magically called by the Eloquent Query Builder class. Our first parameter is a required one that all scopes must include. It’s the Query Builder query so you can access the Query Builder, and return it to chain queries. Next up in our parameters is the UUID, enough said. Thirdly, the boolean parameter to determine whether we return the found model, or the Query Builder instance. This defaults to true as a convenience – it works well for me but feel free to change or remove it if you find it not to your taste.

First up in our method is a bit of a doozy – first we need to check to see if the supplied UUID is a string and secondly we need to make sure it’s a valid UUID. The is_string (line 14) check, while defeated easily, is necessary to ensure we don’t pass an object, array, etc off to preg_match and cause a fatal runtime error – that ain’t catchable! The regular expression you see above is designed specifically to match UUIDv4 strings. Certain parts of the UUID string start with specific letters so we do a check to make sure that we don’t have a v1 UUID, for example.

If for some reason the above regex doesn’t work for you, use ^[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}$ instead, which is from the above UUID library. This regex will verify that a string matches any type of UUID, regardless of version.

If either one of the if statement checks fails, we throw a ModelNotFoundException exception so the developer can reliably catch it. Next up, we simply chain a where method call (line 18), which essentially means WHERE uuid = $uuid (Eloquent actually uses parameters with SQL, but this statement is purely an example of what the query would look likenever ever ever send user data blindly into an SQL query).

Lastly, in our little scope that could, we do a check to see if the $first parameter is set to true or false (line 20). If it’s true, return the firstOrFail method that executes the $search query or return the Query Builder instance so we can keep on chaining. There we are! Our scopeUuid method is done, now it’s on to the scopeIdOrUuid method.

This method is rather similar (the only parameter that’s different is $id_or_uuid – pretty self explanatory really) so I’ll just breeze over it quickly. First, we’ll do a check to ensure it’s either a string, or numeric (whether that be 1 or '1' – line 14), and throw ModelNotFoundException if it isn’t. Next, use regular expressions to determine whether the $id_or_uuid variable is an integer or a UUID (line 18). If it doesn’t, throw a ModelNotFoundException exception.

Next up is a nifty feature of the where method in Query Builder. Did you know where can actually be a callback, you can nest WHERE queries in one bigger WHERE query so that you don’t accidentally break your tightly scoped query with a misplaced WHERE. We’ll use an anonymous function for our callback and use the use language construct so that our anonymous function can inherit the $id_or_uuid variable from the parent scope (line 22).

Once again, a final check to see if the developer wants the model (or catch a ModelNotFoundException) or the Query Builder instance returned to them (line 27).

And there we are! Now you can go out and use it! Oh wait, no you can’t. Unless you’ve already generated UUID fields your models won’t be able to store their UUIDs! Whip up a migration using php artisan make:migration _migration_name_here and let’s add UUIDs to the tables. For each table that needs a UUID, add this to your schema:

$table->string('uuid', 36)->unique();

If you’ve installed doctrine/dbal and are using MySQL, you can also append ->after(‘id’) to the above command so it appears after the ID in your SQL table.

To use this handy trait, use it in a model like so:

// app/User.php

use App/Traits/UuidModel;

class User extends Model
{
      use UuidModel;
// ...

Simple as that! It’s also recommended that you add the ID to the hidden attributes property so the ID is removed from the array/JSON representation of the model. Add the following to all your models using the UUID trait:

/**
* The attributes that should be hidden in array/JSON representations of the model.
*
* @var array
*/
protected $hidden = [
      'id'
];

When it comes to actually using the trait in your application you can use it like any other Eloquent or Query Builder method. An example of this could be a route like so:

get('/user/{user_uuid}', 'UserController@getUser');

And in the UserController class you could do something like this:

public function getUser($user_uuid)
{
      try {
            $user = User::uuid($user_uuid);
      } catch (ModelNotFoundException $e) {
            abort(404);
      }

      // ...

There we are folks, an easy way to implement UUIDs on your Eloquent models in Laravel. If anything should change in the future I’ll do my best to update the article as the changes come.

Click here to see the UuidModel trait in its entirety.


Using X-XSRF-TOKEN HTTP Headers for AJAX in Laravel 5 (Updated)

Update (24/02/2015): Laravel 5.0.6 has been updated to support cleartext X-XSRF-TOKENs. As explained in the recent post CSRF Protection in Laravel explained by Barry vd. Heuvel, Laravel can now process X-XSRF-TOKENs if they are transmitted in cleartext. Some would argue it’s still better to encrypt the CSRF token, but that’s for much smarter InfoSec people than me.

The following article was written for Laravel 5.0.5 in mind, but is still relevant as of 5.0.6

If you’ve recently started using Laravel 5 and are trying to use csrf_token() with the header X-XSRF-TOKEN with your AJAX requests, you’ll notice that you get a HTTP Error code 500, rather than a 200 OK response. This is because the CSRF middleware is expecting the csrf_token via X-XSRF-TOKEN to be encrypted – Something the Laravel documentation doesn’t make clear.

When I originally stumbled across this issue I thought it was a bug in Laravel and submitted a PR (which turned out to be a bad, naughty, terrible, not so good thing to do – in short, I should learn to search.) Regardless, we have two ways of getting around this. Our first way is to just encrypt the damn CSRF token and use that in our code, or alter the middleware to not perform decryption on the CSRF Token.

Option 1 – Encrypted CSRF Token

Our first option is to encrypt the CSRF token. As you may already know, you can access the CSRF token by using the function csrf_token. Load up your routes.php file so we can add the encrypted token to the views.

For each view you call, you’ll need to append this method:

withEncryptedCsrfToken(Crypt::encrypt(csrf_token()));

So, if you were calling a view for the home template, you’d do this:

view('home')->withEncryptedCsrfToken(Crypt::encrypt(csrf_token()));

Terrific. In that template you can access the variable like below:

<meta name="csrf_token" ="{{ $encrypted_csrf_token }}" />

Chuck that in your main view in the <head> so your JavaScript framework of choice can gobble it up. Just make sure to do use Crypt; if you’re in a different namespace.

Option 2 – Non-encrypted CSRF Token

Our second option is to alter the VerifyCsrfToken middleware to not expect an encrypted CSRF Token when transmitted via a HTTP Header.

Open up the VerifyCsrfToken.php middleware (located at app/Http/Middleware/) and we’ll extend the method tokensMatch.

protected function tokensMatch($request)
{
      $token = $request->session()->token();
     
      $header = $request->header('X-XSRF-TOKEN');
     
      return StringUtils::equals($token, $request->input('_token')) ||
             ($header && StringUtils::equals($token, $header));
}

Essentially, what I’ve done is copied the method from Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php then removed the call to $this->encrypter. You’ll also need to add a use at the top of VerifyCsrfToken.php like so:

use Symfony\Component\Security\Core\Util\StringUtils;

Once you’ve done that, you can safely use plain old csrf_token in your X-XSRF-TOKEN header and get 200 - OK with all your AJAX calls. If you didn’t quite figure out the middleware alteration, load up this Gist to see how I modified the VerifyCsrfToken middleware.

Implementing in jQuery

If you happen to be using jQuery with Laravel, here’s how you can add the HTTP Header to your AJAX requests. As usual, there’s a few different options. If you’re doing a heap of requests over the lifetime of the session, you’ll want to set this token for all AJAX requests. If not, you can do it inline with the AJAX call.

First up, the pre-filter to make this global for all $.ajax requests:

$.ajaxPrefilter(function(options, originalOptions, xhr) {
      var token = $('meta[name="csrf_token"]').attr('content');
     
      if (token) {
            return xhr.setRequestHeader('X-XSRF-TOKEN', token);
      }
});

Good stuff. Now, all $.ajax requests in the application lifecycle will use that prefilter with the token and HTTP Header.

If you just need the HTTP Header for one or two requests, here’s how you can add it to the $.ajax call:

$.ajax({
      url: 'http://example.com/api',
      beforeSend: function (xhr) {
            var token = $('meta[name="csrf_token"]').attr('content');
           
            if (token) {
                  return xhr.setRequestHeader('X-XSRF-TOKEN', token);
            }
      },
      /* ... */
});

That “pre-filter” will be in effect for that $.ajax call only.

Moving Forward

Now, it’s entirely up to you how to proceed. Just to be safe, I’ve decided to go with Option 1 because I want to err on the side of caution, but if your Laravel 5 app is super simple and can’t do much/any harm, I think it’s OK for your CSRF Token to match in a string-to-string comparison, but not be valid JSON.

Only time will tell.

Updates:

  • 19/02/2015: — I originally had the CSRF Token encrypted in the boot method of the AppServiceProvider. This was incorrect as csrf_token isn’t set unless it’s called from within a Route. My mistake!
  • 24/02/2015: — Updated with comments about Laravel 5.0.6 now supporting cleartext X-XSRF-TOKENs.

OS X Server and Heartbleed

Heartbleed OS X

If you keep up with technical news you would’ve seen an abundance of articles on something called Heartbleed (or CVE-2014-0160). Essentially, it’s a protocol implementation bug that affects newer versions of OpenSSL, which is used in a majority of Linux installations and can affect many services like Apache, nginx and even IMAP (just to name a few). I’m not going to go into the details because I’ll leave that to the experts, but I highly recommend taking a look at the Heartbleed website to learn more.

Now, when I saw this I got immediately worried about all my clients’ (not to mention my own) installation of OS X Server. Thankfully, OS X has an older implementation of OpenSSL (on 10.9.2 it’s 0.9.8y – found by doing this command in Terminal: openssl version) and according to the Heartbleed website, is not vulnerable. Just to make sure, I ran a Python script found online written by Jared Stafford (download) and when against my server at home, I get the following:

Connecting...
Sending Client Hello...
Waiting for Server Hello...
... received message: type = 22, ver = 0301, length = 53
... received message: type = 22, ver = 0301, length = 2551
... received message: type = 22, ver = 0301, length = 525
... received message: type = 22, ver = 0301, length = 4
Sending heartbeat request...
... received message: type = 21, ver = 0302, length = 2
Received alert:
0000: 02 46                  .F

Server returned error, likely not vulnerable

Further testing on OS X has revealed that Heartbleed won’t be exposing anything, which is a huge relief for me. Having said that, as this was undiscovered for 2 years, theoretically there’s nothing stopping there being another vulnerability in the wild that could be causing the same (or more or less) damage as Heartbleed.

If you run the Python script and find you are affected, I urge you to patch your OpenSSL installation and regenerate your SSL certificates from scratch. As this vulnerability grabs 64KB worth of data from memory, it’s possible your private key could be in that 64KB. This means you need more than just a CSR, you’ll need to start from the beginning. You can check yourself using an excellent service by Filippo Valsorda.

Good luck out there.


SSL Certificate Expiry Monitoring with Nagios

The expiration of an SSL certificate can render your secured service inoperable. In late September, LightSpeed’s SSL certificate for their activation server expired which made any install or update fail due to not being able to connect to said activation server via SSL. This problem lasted two days and was a problem worldwide. This could have been avoided by monitoring the SSL certificate using a monitoring solution like GroundWork. We currently have a script that checks certificates, but it’s only designed to look at certificates on OS X in /etc/certificates. I’ve since written a bash script that will obtain any web-accessible certificate via openssl and check its start and end expiry dates.

The command I use to get the SSL certificate and its expiry is as follows:

echo "QUIT" | openssl s_client -connect apple.com:443 2>/dev/null | openssl x509 -noout -startdate 2>/dev/null

First, we echo QUIT and pipe it to the openssl s_client -connect command to send a QUIT command to the server so openssl will finish up neatly (and won’t expect a ^C to quit the process). Next, that result is piped to openssl x509 which displays certificate information. I also send the noout flag to not prevent the output of the encoded version of the certificate. Finally, startdate or enddate depending on which date I want. With both of the openssl commands I redirect stderr (standard error) output to /dev/null because I’d rather ignore any errors at this time.

The rest of the script performs regex searches of the output (using grep), then formats the timestamp with date before doing a differential of the expiration timestamp, and the current timestamp (both of these are in UNIX Epoch time). Depending on how many you days in the expiration warning, if the number of seconds for the current Epoch time minus the expiration date is less than the expiration day count, a warning is thrown.

To use this script, all you need to specify is the address of the host, the port and number of days to throw a warning (e.g. specify 7 if you want a warning period of 7 days). For example, to get the SSL certificate for apple.com on the port 443 you would enter:

./check_ssl_certificate.expiry -h apple.com -p 443 -e 7

And you should get something like below:

OK - Certificate expires in 338 days.

Not only does this script verify the end date of the certificate, but it also verifies the start date of the certificate. For example, if your certificate was set to be valid from 30th December 2013, you might see something like the error below:

CRITICAL - Certificate is not valid for 30 days!

Or if your certificate has expired, you’ll see something like below:

CRITICAL - Certificate expired 18 days ago!

Hopefully this script will prevent your certificates from expiring without your knowledge, and you won’t have angry customers that can’t access your service(s)!

Check out the code on GitHub!


PSA: Fake ATO Refund Phishing Email

This morning I received an email purportedly from the Australian Tax Office (ATO) which, at first, excited me (I had just woken up) because I was getting another refund! Hooray! But… there’s always a catch. Upon actually reading the email I noticed several red flags which made me realise that the email is in fact, a phishing email.

Tax Refund Scam
The phishing email as seen on my Mac.

First off, the refund amount is listed as AUD. Given the Australian Tax Office operates only in Australia, there is no reason that they would have to specify what currency the refund amount is in, are they all of a sudden going to refund in USD, EUR? I don’t think so. Secondly, the website is cloudaccess.net, not ato.gov.au. The Australian Tax Office would never use another domain for any of their pages.

The last paragraph makes mention of an SSL certificate, yet the resource for the aforementioned link is http, not https (which all secure websites should be). Given how the banks have been training people to look out for the lock for some years now I think this would be a obvious red flag for many people.

The goofballs who made the email even used @ato.com.au as the reply email address. Any Australian resident who uses the internet should know that the Australian Tax Office is a government branch, and will use the .gov.au TLD, not .com.au

For more details regarding the Australian Tax Office and online security, I recommend checking out the ATO’s article titled “Online Security” as it contains examples of past scams, and how to identify a fraudulant email.

Stay safe out there!