DIY Drupal hosting

The meta stuff

I’ll be writing a series of posts exploring DIY drupal hosting options. Many agencies and freelancers need a workflow to manage and maintain their Drupal sites on a budget. Of course, they incur the cost of maintaining and deploying the system(at least initially) and the additional learning curve involved in using the system, but they get the following advantages:

  • More control over the hosted sites. It is easy to create and deploy new environments to demo sites to clients, triage bugs, run tests etc.
  • Better workflow management. Site admins can run security updates or revert/update modules and features across sites, take periodic backups, restore them and migrate sites in and out of the hosting system. This is an important factor for developers if you’re running an agency supporting lots of sites.
  • Save $$$. Though having a DIY solution has its share of initial hiccups, it could save money in the long run.

There are lot of wonderful solutions, most notably PantheonOmega8 and Acquia Cloud. By all means, go ahead and try them out and see how they fit your use case. These posts are intended to educate about other tailor-made options which can be built and modified to fit your workflow.

TLDR; You always trade headache for money.

What to look for

With the meta stuff out of the way, I’ll elaborate here on what features I look for when shopping a DIY hosting solution. These cover 80% of the use cases needed by most organizations and/or developers. Please get in touch with me if any major use case is not listed here.

  • Ability to spin off different staging and testing environments of the site quickly. Many a times, we have to quickly create a version of the site with a feature or 2 added, for demo-ing, or create a quick production replica to triage a bug. Our solution should be able to create such throwaway environments with little effort.
  • dev production parity. The throwaway setups mentioned above should be as close to the production setup as possible. For instance, the throwaway setup should also have HTTPS if the production site has one. That’s how close “close” means.
  • Secure. The DIY solution should have an ACL and the hosted environment should have proper protocols in place, like allowing SSH access only from trusted networks and users, firewall configuration etc. Note that this has nothing to do with the security of the site hosted on the platform. That’s a post for another day!
  • truly agile. Deployment to production should be a predictable, documented and repeatable activity. This opens up the possibility of automating deployment. Once deployments stop becoming time sinks, as a consequence, frequent deployments and iterations happen. This naturally facilitates continuous delivery in our workflow.

Secondary traits

We can have a working DIY solution with just the above essential qualities. These are other features which make the above main features easier and aid them. Some of them might be Drupal specific. These are between essential and nice-to-have, and make our DIY solution more complete.

  • Extensible. I should be able to build my own custom LAMP stack if the site needs it. To elaborate, I should be able to deploy site A and site B running different versions of PHP, or one using Nginx and the other using Apache. Another example is talking to different systems. For example, every new git branching should trigger the system to create a new environment from scratch, with sensible defaults. The common thread here is, the solution should be extensible.
  • Ease of installation We already inherit lot of complexity by opting for a DIY Drupal solution. Setting up one from scratch should be as smooth as possible.
  • Plays well with Drupal The solution works with Drupal out of the box, with little tinkering. One example would be allowing drush usage. A good DIY solution would ideally have provision for drush site aliases. Another example, is taking into consideration the fact that Drupal isn’t a complete 12 factor app. Any platform which assumes otherwise needs a lot of tinkering to keep the lights on.
  • Usability. There needs to be a commandline tool or a UI dashboard to manage the sites, maintain the system etc., despite the fact that the intended audience for this solution are mostly developers. What if your project manager wants to demo a work-in-progress feature to a client while you’re away? 😉
  • Drupal 8. This goes without saying, but deserves a mention. Lot of ink has been spilled on how Drupal 8 is a radical shift in development from its predecessors. Our solution needs to take this into account to stay relevant in future.

The nice-to-haves

  • Support for addon services. It would be convinient if the DIY solution supports other addons commonly used for Drupal sites, like Redis, Memcached and Solr.
  • Support for other frameworks/CMSes. If you are an agency/developer working on multiple frameworks, this might be the deal breaker for you. It would be great if our DIY solution supports other systems like WordPress, Rails, Symfony etc., though it is not a mandate.

Assumptions and expectations

The solutions we will be discussing are tried and tested on Linux-ish systems(more specifically, Ubuntu and flavours). There is no guarantee that this will work if you’re using Windows to build it. Also, if you are running Drupal on IIS or Microsoft SQL server, this setup won’t work for you.

With this premise, we will be walking through 3 DIY Drupal hosting setups in the subsequent posts which you can try at home!

DIY Drupal hosting series

Introduction to DIY Drupal hosting – you’re here

DIY Drupal hosting using Aegir

DIY Drupal hosting using OpenDevShop

DIY Drupal hosting using DrupalVM

Entity validation in Drupal 8 – part 3

We’ve seen how validation works and how to create a custom validation component previously. Chances are, a validation component already exists for most of the requirements. Thanks to Composer and the way Symfony is organized as components, it is easy to reuse existing components. We will try our hand at one such component, the Zip Code validator.

We will use this validator for one of our text fields. In order to do this, we have to first add this validator to our composer.json file. Here’s how our top level composer.json require section looks before adding any external library.

// ...
"require": {
    "composer/installers": "^1.0.21",
    "wikimedia/composer-merge-plugin": "~1.3"
},

We add zip code validator component as a dependency to our Drupal setup using this command, as mentioned in the README.

$ composer require barbieswimcrew/zip-code-validator

After the library is installed successfully, here’s how the updated composer.json looks like.

"require": {
    "composer/installers": "^1.0.21",
    "wikimedia/composer-merge-plugin": "~1.3",
    "barbieswimcrew/zip-code-validator": "^1.0"
},

We can’t use a Symfony validation component by directly invoking it. So, we have to do a little prepping to bring it into Drupal’s context. The first thing we do towards this direction is to convert zip code validation into a Drupal plugin, similar to the ShirtSize plugin we created previously.

namespace Drupal\custom_validation\Plugin\Validation\Constraint;

use ZipCodeValidator\Constraints\ZipCode;

/**
 * Zipcode constraint.
 *
 * An example of using an external validation component for zip codes.
 *
 * @Constraint(
 *   id = "ZipCode",
 *   label = @Translation("Zipcode constraint", context = "Validation")
 * )
 */
class ZipCodeConstraint extends ZipCode {

  public $message = 'This zip code is not valid.';

  /**
   * {@inheritdoc}
   */
  public function validatedBy() {
    return '\ZipCodeValidator\Constraints\ZipCodeValidator';
  }

}

This is just a thin layer on top of our existing validator, just to “drupalize” our validator. Now, we are ready to use it on our entity fields. Assuming you have a plain text field that goes by the machine name field_zip_code, here’s how you would use the zip code validator.

/**
 * Implements hook_entity_bundle_field_info_alter().
 */
function custom_validation_entity_bundle_field_info_alter(&$fields, \Drupal\Core\Entity\EntityTypeInterface $entity_type, $bundle) {
  if ($entity_type->id() == 'node' && $bundle == 'article') {
    $fields['field_zip_code']->addConstraint('ZipCode', ['iso' => 'IN']);
  }
}

The iso parameter is the country for which the zip code needs to be validated, which is India in this case. The validator supports lot of countries which you can try out.

Let’s take this new field validation for a spin.

use Drupal\node\Entity\Node;

$node = Node::create([ 'title' => 'New article with an invalid zip code', 'type' => 'article']);
$node->uid = 1;
$node->field_zip_code = 'foo'; // bad zip code
$violations = $node->validate();
if ($violations->count() > 0) {
  foreach($violations as $violation) {
    print_r($violation->getMessage()->render());
    print("\n");
    print_r($violation->getPropertyPath());
    print("\n");
  }
 } else {
  $node->save();
}

And our validator rings the alarm.

$ drush @dev scr f.php
[warning] preg_match() expects parameter 2 to be string, object given ZipCodeValidator.php:254
This zip code is not valid.
field_zip_code

Sounds good. Wait, what’s that warning about preg_match? Its got to do with mismatch between Symfony’s entity validation and Drupal’s validation mechanism. Here’s how the validation code looks like inside Zipcode’s validator:

if (!preg_match("/^{$pattern}$/", $value, $matches)) {
  // ...
}

Here, $value is expected to be a string but Drupal passes a field object to the validate function, thus making the pregmatch fail for all cases. We can address this by overriding the Validator. Let’s write a Drupal layer over the existing validator which strips the bare value from the field in question and pass it to the parent validator.

namespace Drupal\custom_validation\Plugin\Validation\Constraint;

use Symfony\Component\Validator\Constraint;
use ZipCodeValidator\Constraints\ZipCodeValidator;
/**
 * Validates zip codes.
 */
class ZipCodeConstraintValidator extends ZipCodeValidator {

  /**
   * {@inheritdoc}
   */
  public function validate($field, Constraint $constraint) {
    $value = $field->value;
    parent::validate($value, $constraint);
  }
}

The naming convention for validators is the ‘Validator’ string suffixed to the constraint class name, so we can drop the validatedBy function from ZipCodeConstraint. Run this new validator to get correct results this time.

$node = Node::create([ 'title' => 'New article - clean', 'type' => 'article']);
$node->uid = 1;
// make sure you give a correct zip code according
// to the country you specified in the validation constraint
$node->field_zip_code = '600044';
$violations = $node->validate();
if ($violations->count() > 0) {
  foreach($violations as $violation) {
    print_r($violation->getMessage()->render());
    print("\n");
    print_r($violation->getPropertyPath());
    print("\n");
  }
 } else {
  $node->save();
 }

As noted in the previous article, this validation constraint works even via the UI.

Drupal entity validation of Zip Code
Drupal entity validation of Zip Code

The code for this post can be checked out from the external-validator tag of the custom validation module repo.

$ git clone git@github.com:drupal8book/custom_validation.git
$ cd custom_validation
$ git checkout -f external-validator

Entity validation in Drupal 8 – part 2

In the first part, we saw how entity validation works in Drupal 8, why it is a separate component and how most parts are adopted from Symfony’s entity validation framework. We will try our hands on creating our own custom validator in this post.This validator will fail to create a node if the user ID is not specified. In other words, prevent creation of anonymous nodes.

We’ve already seen that constraints are Drupal 8 plugins. Let’s go ahead and create them by checking out tag entity-level-validation of the repo(fastest), using Drupal Console(fast) or manually(slowest). Either ways, we have 2 classes, the Constraint class which holds the metadata of the constraint.

namespace Drupal\custom_validation\Plugin\Validation\Constraint;

use Drupal\Core\Entity\Plugin\Validation\Constraint\CompositeConstraintBase;

/**
 * Prevent anon nodes from being created.
 *
 * @Constraint(
 *   id = "PreventAnon",
 *   label = @Translation("Prevent nodes being created by anon users", context = "Validation"),
 *   type = "entity:node"
 * )
 */
class PreventAnonConstraint extends CompositeConstraintBase {

  /**
   * Message shown when an anonymous node is being created.
   *
   * @var string
   */
  public $message = 'Cannot create a node that does not belong to any user.';

  /**
   * {@inheritdoc}
   */
  public function coversFields() {
    return ['uid'];
  }
}

Note that PreventAnonConstraint extends CompositeConstraintBase, which should be the case for constraints at the entity level. We shall see the reason for this shortly. The constaint class also indicates what fields/properties this constraint applies to(just the uid in our example).

…and the validator class which does the actual validation.

namespace Drupal\custom_validation\Plugin\Validation\Constraint;

use Symfony\Component\Validator\Constraint;
use Symfony\Component\Validator\ConstraintValidator;

/**
 * Validates the PreventAnon constraint.
 */
class PreventAnonConstraintValidator extends ConstraintValidator {

  /**
   * {@inheritdoc}
   */
  public function validate($entity, Constraint $constraint) {
    if (!isset($entity)) {
      return;
    }
    if (!$entity->getOwnerId()) {
      $this->context->addViolation($constraint->message);
    }
  }
}

The constraint does not take effect until we add it using hooks in our module.

/**
 * Implements hook_entity_type_alter().
 */
function custom_validation_entity_type_alter(array &$entity_types) {
  $node = $entity_types['node'];
  $node->addConstraint('PreventAnon', []);
}

And now, to test out our custom validator.

use Drupal\node\Entity\Node;

$node = Node::create([ 'title' => 'New article', 'type' => 'article']);
$violations = $node->validate();
if ($violations->count() > 0) {
  foreach($violations as $violation) {
    print_r($violation->getMessage()->render());
    print("\n");
    print_r($violation->getPropertyPath());
    print("\n");
  }
}

Here’s what we get,

$ drush scr node-validate.php
Cannot create a node that does not belong to any user.
uid

Field level validation

Let’s add a custom validator at the field level now. To add a dash of variety, we will also pass arguments to our validator, as to what set of values the field can hold. Imagine I’m running an e-commerce store and I’m having a field called shirt size, which can hold only the following set of values: XXS, XS, S, M, L, XL and XXL.

Check out tag field-level-validation from the repo or create your own using drupal console.

First, the constraint class.

namespace Drupal\custom_validation\Plugin\Validation\Constraint;

use Symfony\Component\Validator\Constraint;
use Symfony\Component\Validator\Exception\MissingOptionsException;

/**
 * Shirt sizes.
 *
 * @Constraint(
 *   id = "ShirtSize",
 *   label = @Translation("Shirt size constraint", context = "Validation"),
 * )
 */
class ShirtSizeConstraint extends Constraint {

  /**
   * @var array
   */
  public $sizes;

  /**
   * Message shown when shirt size is incorrect.
   *
   * @var string
   */
  public $message = '(%shirt_size) is an invalid shirt size.';

    /**
     * ShirtSize constructor.
     * @param mixed $options
     */
    public function __construct($options = null)
    {
        if (null !== $options && !is_array($options)) {
            $options = array(
                'sizes' => $options
            );
        }
        parent::__construct($options);
        if (null === $this->sizes) {
            throw new MissingOptionsException(sprintf('The option "sizes" must be given for constraint %s', __CLASS__), ['sizes']);
        }
    }
}

We can pass various allowed shirt sizes as constructor arguments while initializing the constraint.

Next, create a dropdown field(list of text) named field_shirt_size for the article content type. The validator class looks like this:

namespace Drupal\custom_validation\Plugin\Validation\Constraint;

use Symfony\Component\Validator\Constraint;
use Symfony\Component\Validator\ConstraintValidator;

/**
 * Validates the ShirtSize constraint.
 */
class ShirtSizeConstraintValidator extends ConstraintValidator {

  /**
   * {@inheritdoc}
   */
  public function validate($field, Constraint $constraint) {
    $value = $field->value;
    if (!isset($value)) {
      return NULL;
    }
    if (!in_array($value, $constraint->sizes)) {
        $this->context->addViolation($constraint->message, array('%shirt_size' => $value));
    }
  }
}

A fairly simple validator. It checks whether the given field value is in the list of values given in the constraint. Let’s apply the constraint to the specific field and bundle.

/**
 * Implements hook_entity_bundle_field_info_alter().
 */
function custom_validation_entity_bundle_field_info_alter(&$fields, \Drupal\Core\Entity\EntityTypeInterface $entity_type, $bundle) {
  if ($entity_type->id() == 'node' && $bundle == 'article') {
    $fields['field_shirt_size']->addConstraint('ShirtSize', ['sizes' => ['M','L','XL']]);
  }
}

We effectively allow only “M”, “L” and “XL” sizes. Let’s quickly try to create a node with an invalid size.

$node = Node::create([ 'title' => 'New article - clean', 'type' => 'article']);
$node->uid = 1;
$node->field_shirt_size = 'XS';
$violations = $node->validate();
if ($violations->count() > 0) {
  foreach($violations as $violation) {
    print_r($violation->getMessage()->render());
    print("\n");
    print_r($violation->getPropertyPath());
    print("\n");
  }
} else {
  $node->save();
}

As “XS” is not in our allowed values list, it shoots an error.

$ drush @dev scr f.php
(<em class="placeholder">XS</em>) is an invalid shirt size.
field_shirt_size

I’m not too sure where the <em> tags are coming from, but our custom validation works good enough. This takes precedence even while creating nodes from the node add form. Let’s confirm this by creating an article node via UI with an invalid value for shirt size.

In the next part, we shall wrap up the remaining bits and pieces of Drupal 8 entity validation.

Entity validation in Drupal 8 – part 1 – how validation works

Drupal 8 has its entity validation separate and decoupled from the typical validation given by its form API. This is done for a lot of reasons. For one, entities might get added from other non UI means, like via the REST API, or programmatically, while importing data from an external source. Under these circumstances, the entity validation API comes in handy.

Drupal 8’s validation API uses the Symfony validator component.Each validation mechanism can be at the entity level(composite), field level or entity property level. Validation can be specified by multiple means.

1.While creating the entity as a part of the annotation.

Ex: the Comment entity has a validation constraint which imposes a restriction where the name of the anonymous comment author cannot match the name of any registered user. This is implemented using CommentNameConstraint and specified in the Comment entity annotation.

 *   bundle_entity_type = "comment_type",
 *   field_ui_base_route  = "entity.comment_type.edit_form",
 *   constraints = {
 *     "CommentName" = {}
 *   }
 * )
 */
class Comment extends ContentEntityBase implements CommentInterface {

2.Inside the entity class’s baseFieldDefinitions().

Ex: The User entity has a constraint where each user name should be a unique value.

$fields['name'] = BaseFieldDefinition::create('string')
  ->setLabel(t('Name'))
  ->setDescription(t('The name of this user.'))
  ->setRequired(TRUE)
  ->setConstraints(array(
    // No Length constraint here because the UserName constraint also covers
    // that.
    'UserName' => array(),
    'UserNameUnique' => array(),
  ));

We will see what BaseFieldDefinition means in a future post. For now, all you have to understand is, the above line places a validation constraint that the name property of every user object should be unique.

3.Entity validation constraints can be placed on existing entities from other modules via hooks.

This implements hook_entity_type_alter.

function my_module_name_entity_type_alter(array &$entity_types) {
  $node = $entity_types['node'];
  $node->addConstraint('CustomPluginName', ['plugin', 'options']);
}

We shall be creating one such validation constraint on the node entity shortly.

A validation component consists of 2 parts.

The constraint contains the metadata/rules required for the validation, the messages to show as to what exactly got invalidated, and a pointer to the validation class, whose default value is a “Validator” string appended to the fully qualified constraint class name.

/**
 * Returns the name of the class that validates this constraint.
 *
 * By default, this is the fully qualified name of the constraint class
 * suffixed with "Validator". You can override this method to change that
 * behaviour.
 *
 * @return string
 */
public function validatedBy()
{
    return get_class($this).'Validator';
}

The validation class contains the actual validation implementation. For example, a “unique name” constraint’s validator will iterate through all entities in the database to ensure that the name of the entity being validated is not used by any other entity. The validator class also has access to the constraint class metadata, messages etc. It should, at minimum, implement the validate method, which takes in the object to be validated(string, entity etc.) and the associated constraint. Upon failing the validation, this method returns an object of type ConstraintViolationInterface. This gives all the information as to why the validation failed, where exactly it failed, the invalid value etc.

Let’s see how a node can be validated and the validation errors consumed with the below example.

use Drupal\node\Entity\Node;

$node = Node::create([ 'title' => 'New article', 'type' => 'article']);
$node->field_email = 'foobar';
$violations = $node->validate();
if ($violations->count() > 0) {
  foreach($violations as $violation) {
    print_r($violation->getMessage()->render());
    print("\n");
    print_r($violation->getPropertyPath());
    print("\n");
    print_r($violation->getInvalidValue());
    print("\n");
  }
 }

Assuming you have an email field which goes by the machine name field_email, if you run this code using drush scr command in a Drupal 8 setup, your output should be very similar to this.

$ drush scr node-validate.php
This value is not a valid email address.
field_email.0.value
foobar

The getPropertyPath give the field name and the delta as to where the violation occurs.

Now that we got a hang of how entity validation works, let’s create our own validation constraint in the next post.