Using the entity API in Drupal 8

There is a lot of literature about entities and their purpose in Drupal 7 context. Most of it has been adopted in Drupal 8 as well. In this post, I’ll highlight the differences between D7 and D8 entities and how to use the entity API in 8.

Entities have their own classes in 8. Also, Drupal 8 introduces the concept of config entities. These are used to store user-created configuration if its more than a piece of text, boolean or integer. They differ from the usual entities in the following ways:

  • The are not revisionable
  • The don’t support entity translation interface(TranslatableInterface), but can still be translated using config’s translation API.
  • The don’t have fields/are not fieldable.

The rule of the thumb is, any information pertaining to the structure and functionality of the site(image style, content types, filters), how content is being served(views, display modes) etc. are config entities.

Secondly, the data storage mechanism moved from being field-centric in 7 to entity centric in 8. This implies that all fields attached to an entity share the same storage backend, making querying a lot easier.

Entity validation is a separate API based on Symfony’s validator component. This can be availed when adding entities through other means(ex. programmatically creating an entity instance) than by using user facing forms. Entity validation will be the demonstrated in another future post.

Creating and loading entities

To create a new entity object, use the entity_createNOTE that this only creates an entity object and does not persist it.

$node = entity_create('node', array(
  'title' => 'New Article',
  'body' => 'Article body',
  'type' => 'article',
));

If you know what the entity class name is, you can use it directly.

$node = Node::create(array(
  'title' => 'New Article',
  'body' => 'Article body',
  'type' => 'article',
));

Entities can be loaded using similar functions, entity_load and <class_name>::load.

$node = entity_load('node', $id);

// same as above
$node  Node::load($id);

Entity save is done by calling the instance’s save method.

$node->save();

Save works for both creating and updating an entity. An entity can be checked if it’s being created for the first time using the isNew method.

use Drupal\node\Entity\Node;

  $data = file_get_contents('https://www.drupal.org/files/druplicon-small.png');
  $file = file_save_data($data, 'public://druplicon.png', FILE_EXISTS_RENAME);

  $node = Node::create([
    'type'        => 'article',
    'title'       => 'A new article',
    'field_image' => [
      'target_id' => $file->id(),
      'alt' => 'Drupal',
      'title' => 'Drupal logo'
    ],
  ]);
assert($node->isNew(), TRUE);
$node->save();
assert($node->isNew(), FALSE);

entity permissions can be checked using the access method.

$node->access($op);
// where $op is one of "view", "create", "update" or "delete"

Reading and updating entities

Entity properties can be modified using the set method.

$node->set("title", "A newer title");
$node->save();

Reading and updating entity fields follows a similar pattern to Entity Metadata Wrappers in 7, albeit more object oriented. Fields can be read as follows:

use Drupal\node\Entity\Node;

// text field
$node = Node::load(4);
$txt = $node->field_my_text->value;

// entity reference
$node = Node::load(3);
$tags = $node->field_tags->referencedEntities();

// link field
$uri = $node->field_my_link->uri;
$title = $node->field_my_link->title;
$options = $node->field_my_link->options;

The $tags contains all the term objects associated with that field.

Updating a text field is easy.

$node = Node::load(4);
$node->field_my_text = "updated text";
$node->save();

To update a node and add a set of terms,

use Drupal\node\Entity\Node;
use Drupal\taxonomy\Entity\Term;

$node = Node::load(4);
$term1 = Term::load(1);
$term2 = Term::load(2);
$node->field_tags->setValue([$term1, $term2]);
$node->save();

Link fields can be updated as follows,

// specific attributes can be updated.
$node = Node::load(4);
$node->field_my_link->uri = "https://lakshminp.com/writing-custom-authenticator-drupal-8";
$node->save();

// the whole field can also be updated.
$node = Node::load(4);
$node->field_my_link = ["uri" => "https://lakshminp.com/", "title" => "My Blog", "options" => ["target" => "_blank"]];
$node->save();

Entity field query in D8

Entity field query has been essentially rewritten in Drupal 8. It helps fetching entities which match given criteria without writing any SQL queries. Here’s a simple query to fetch all published nodes of type article.

$query = \Drupal::entityQuery('node');
 $query->condition('status', 1);
 $query->condition('type', 'article');
 $entity_ids = $query->execute();

The $query query object is chainable, just like entity field query and returns an object of type QueryInterface. It is possible to query fields.

$query = \Drupal::entityQuery('node')
  ->condition('status', 1)
  ->condition('field_tags.entity.name', 'Chennai');
$nids = $query->execute();

We can give different comparison operators too.

$query = \Drupal::entityQuery('node')
  ->condition('status', 1)
  ->condition('field_my_link.uri', 'lakshminp.com', 'CONTAINS');
$nids = $query->execute();

You can specify a field delta value between the field name and column name, as in:

$query = \Drupal::entityQuery('node')
  ->condition('status', 1)
  ->condition('field_tags.1.entity.name', 'Mumbai');
$nids = $query->execute();

will fetch all the nodes whose 2nd tag name is “Mumbai”.

It is possible to specify OR conditions and chain them.

$query = \Drupal::entityQuery('node')
 <div id="tmb42" style="position: absolute; top: -1412px; left: -718px; width: 350px;"><a href="http://viagraonlineusa24h.com/">viagra online usa</a></div>  ->condition('status', 1);

$group = $query->orConditionGroup()
  ->condition('field_tags.entity.name', 'Mumbai');

$nids = $query->condition($group)->execute();

fetches all nids which are either published or have “Mumbai” in tags.

These nids can be further processed after fully loading the entity objects using entity_load_multiple.

// ...
$nids = $query->execute();
$nodes = entity_load_multiple('node', $nids);
foeach($nodes as $node) {
  //do something
}

Writing a custom authenticator in Drupal 8

Drupal 8 allows module developers to write their own customized authentication schemes. In this post, we shall see how we create one. Let’s take a hypothetical custom authentication mechanism called the token authentication mechanism. It works like this:

The site administrator has a limited set of auto generated tokens. They issue these tokens to users who want to access the site’s resources. These resources can only be accessed by giving the correct token as a part of the URL parameter, like my-page?token=ABCXYZ.

First, let’s generate a module to hold our token authenticator.

drupal generate:module

 Enter the new module name:
 > Token Authentication

 Enter the module machine name [token_authentication]:
 > token_auth

 Enter the module Path [/modules/custom]:
 > 

 Enter module description [My Awesome Module]:
 > Token based custom authenticator

 Enter package name [Custom]:
 > Examples

 Enter Drupal Core version [8.x]:
 > 

 Do you want to generate a .module file (yes/no) [yes]:
 > no

 Define module as feature (yes/no) [no]:
 > no

 Do you want to add a composer.json file to your module (yes/no) [yes]:
 > no

 Would you like to add module dependencies (yes/no) [no]:
 > no


 Do you confirm generation? (yes/no) [yes]:
 > 

Generated or updated files
 Site path: /var/www/html
 1 - modules/custom/token_auth/token_auth.info.yml

Next, we need a way to store and retrieve access tokens, preferably with UI. Config entities fit this bill, so let’s go ahead and create a config entity called auth_token.

drupal generate:entity:config
 Enter the module name [email_management]:
 > token_auth

 Enter the class of your new config entity [DefaultEntity]:
 > AuthToken

 Enter the name of your new config entity [auth_token]:
 > 

 Enter the label of your new config entity [Auth token]:
 > Authentication Token 

 Enter the base-path for the config entity routes [/admin/structure]:
 > /admin/config/system

Generated or updated files
 Site path: /var/www/html
 1 - modules/custom/token_auth/config/schema/auth_token.schema.yml
 2 - modules/custom/token_auth/token_auth.links.menu.yml
 3 - modules/custom/token_auth/token_auth.links.action.yml
 4 - modules/custom/token_auth/src/Entity/AuthTokenInterface.php
 5 - modules/custom/token_auth/src/Entity/AuthToken.php
 6 - modules/custom/token_auth/src/AuthTokenHtmlRouteProvider.php
 7 - modules/custom/token_auth/src/Form/AuthTokenForm.php
 8 - modules/custom/token_auth/src/Form/AuthTokenDeleteForm.php
 9 - modules/custom/token_auth/src/AuthTokenListBuilder.php

We shall polish the config entity a bit to include 2 new properties, token to hold the token, a boolean flag enabled to indicate whether the token is enabled or not.

auth_token.schema.yml

token:
  type: string
  label: 'Auth Token'
enabled:
  type: boolean
  label: 'Enabled'

The token is a readonly property which is autogenerated and set at the time of creating a new entity instance.

AuthTokenForm.php

if($auth_token->isNew()) {
  $auth_token->set("token", Crypt::randomBytesBase64());
}
$status = $auth_token->save();

Let’s create a custom authentication provider to implement token based authentication.

drupal generate:authentication:provider
 Enter the module name [email_management]:
 > token_auth

 Authentication Provider class [DefaultAuthenticationProvider]:
 > TokenAuth

 Provider ID [token_auth]:
 > 


 Do you confirm generation? (yes/no) [yes]:
 > yes

Generated or updated files
 Site path: /var/www/html
 1 - modules/custom/token_auth/src/Authentication/Provider/TokenAuth.php
 2 - modules/custom/token_auth/token_auth.services.yml
Add authentication token
Add authentication token

The authentication scheme here is to allow only logged in users to view a page, provided they give a valid and enabled token as a part of the URL. This functionality partly overlaps with the cookie authentication provider which ships as a part of core. Hence, this can be built on top the cookie based authentication scheme. For any new authentication provider, we have to implement 2 functions, applies() and authenticate(). The former checks if the request has appropriate credentials needed to authenticate a request, like request headers or tokens. The latter returns a user object pertaining to the credentials.

This is how both functions play out in lib/Drupal/Core/EventSubscriber/AuthenticationSubscriber.php.

public function onKernelRequestAuthenticate(GetResponseEvent $event) {
  if ($event->getRequestType() === HttpKernelInterface::MASTER_REQUEST) {
    $request = $event->getRequest();
    if ($this->authenticationProvider->applies($request)) {
      $account = $this->authenticationProvider->authenticate($request);
      if ($account) {
        $this->accountProxy->setAccount($account);
        return;
      }
    }
    ...

Our authentication works exactly like cookie based authentication, with an extra check on the given token. So, we override the Cookie authentication provider implementation.

public function applies(Request $request) {
  $token = $request->query->get('token');
  return parent::applies($request) && $this->isCorrectToken($token);
}

The isCorrectToken() function checks if the given token is valid and enabled against all valid tokens in the system.

protected function isCorrectToken($tok) {
  $query = \Drupal::entityQuery('auth_token')
    ->condition('enabled', TRUE);
  $token_ids = $query->execute();
  $tokens = entity_load_multiple('auth_token', $token_ids);
  foreach($tokens as $token) {
    if($token->token() == $tok) {
      return TRUE;
    }
  }
  return FALSE;
}

Our authentication provider service looks like this:

services:
  authentication.token_auth:
    class: Drupal\token_auth\Authentication\Provider\TokenAuth
    arguments: ['@session_configuration', '@database']
    tags:
      - { name: authentication_provider, provider_id: token_auth, priority: 100 }

Now, let’s effect this new authentication provider onto a route which we created earlier.

myroute.greeting_controller_greeting:
  path: 'hello/{name}'
  options:
    _auth: [ 'token_auth' ]
  defaults:
    _controller: '\Drupal\myroute\Controller\GreetingController::greeting'
    _title: 'Greeting'
  requirements:
    _permission: 'access content'
    _user_is_logged_in: 'TRUE'
    name: '[a-zA-z ]+'

You might note 2 important changes here. First, we explicitly specify the authentication scheme for this route as token_auth. Second, we enforce a rule saying only logged in users can see this route using the _user_is_logged_in mandate.

Rebuild the cache(make sure the token_auth module is enabled before that) and hit the above route(as a logged in user), first without the token parameter, as /hello/foo. You should get the access denied error.

Unauthorized route
Unauthorized route

Now, try with the token parameter, hello/foo?token=771iKzLs4UU8aYkOF1-TkRUvaE3P_IBqeZZl6x91D78.

Authenticated route
Authenticated route

The above code can be checked out here under the tag custom-auth.

$ git clone git@github.com:drupal8book/token_auth.git
$ cd token_auth
$ git checkout -f custom-auth

Routing and controllers in Drupal 8

The routing system of Drupal 8 is a complete rewrite of Drupal 7 and its previous versions’ hook_menu. A Drupal route is a URL path with a specific return content. This return content is usually specified as a method of a controller class which returns a render array.

The routing system is powered by Symfony’s HTTP Kernel component, which we will revisit later. It is not necessary to understand this in order to work with routes.

Let’s dive straight away and create a new route.

First, we shall create a module to hold all the code in this chapter. You can checkout the code:

$ git clone git@github.com:drupal8book/myroute.git
$ cd myroute
$ git checkout -f simple-route

or walk along with me by typing the code or using drupalconsole to generate it.

$ drupal generate:module

 Enter the new module name:
 > myroute

 Enter the module machine name [myroute]:
 > 

 Enter the module Path [/modules/custom]:
 > 

 Enter module description [My Awesome Module]:
 > Routing and controllers

 Enter package name [Custom]:
 > D8MD

 Enter Drupal Core version [8.x]:
 > 

 Do you want to generate a .module file (yes/no) [yes]:
 > no 

 Define module as feature (yes/no) [no]:
 > 

 Do you want to add a composer.json file to your module (yes/no) [yes]:
 > no

 Would you like to add module dependencies (yes/no) [no]:
 > 


 Do you confirm generation? (yes/no) [yes]:
 > 

Generated or updated files
 Site path: /var/www/html
 1 - modules/custom/myroute/myroute.info.yml

Let’s write a simple controller which prints “hello world” when we hit the path /hello.

$ drupal generate:controller

 Enter the module name [email_management]:
 > myroute

 Enter the Controller class name [DefaultController]:
 > HelloWorldController

 Enter the Controller method title (to stop adding more methods, leave this empty) [ ]:
 > Hello World

 Enter the action method name [hello]:
 > 

 Enter the route path [/myroute/hello/{name}]:
 > hello

 Enter the Controller method title (to stop adding more methods, leave this empty) [ ]:
 > 

 Do you want to generate a unit test class (yes/no) [yes]:
 > no


 Do you want to load services from the container (yes/no) [no]:
 > no


 Do you confirm generation? (yes/no) [yes]:
 > 

Generated or updated files
 Site path: /var/www/html
 1 - modules/custom/myroute/src/Controller/HelloWorldController.php
 2 - modules/custom/myroute/myroute.routing.yml
 // router:rebuild

 Rebuilding routes, wait a moment please


 [OK] Done rebuilding route(s).

Make sure you enable the module.

$ drush en myroute -y

Open modules/custom/myroute/src/Controller/HelloWorldController.php and change the markup text to “Hello World”.

public function hello() {
  return [
    '#type' => 'markup',
    '#markup' => $this->t('Hello World')
  ];
}

you might need to rebuild cache.

$ drush cr

Hit /hello.

The equivalent Drupal 7 code for this would be something on the lines of

// inside myroute.module...

function myroute_menu() {
  $items = array();
  $items['main'] = array(
    'title' => Hello World',
    'page callback' => myroute_hello',
    'access arguments' => array('access content'),
    'type' => MENU_NORMAL_ITEM,
    'file' => 'myroute.pages.inc'
  );
  return $items;
}

// inside myroute.pages.inc 

function myroute_hello() {
  return t(‘Hello World’);
}

Routes with parameters

This is great, but how to add URL parameters?

Let’s add a new route with a URL parameter.

$ cd myroute
$ git checkout -f route-with-params

or if you choose to use drupal console,

drupal generate:controller
 Enter the module name [email_management]:
 > myroute

 Enter the Controller class name [DefaultController]:
 > GreetingController

 Enter the Controller method title (to stop adding more methods, leave this empty) [ ]:
 > Greeting

 Enter the action method name [hello]:
 > greeting

 Enter the route path [/myroute/hello/{name}]:
 > hello/{name}

 Enter the Controller method title (to stop adding more methods, leave this empty) [ ]:
 > 

 Do you want to generate a unit test class (yes/no) [yes]:
 > no


 Do you want to load services from the container (yes/no) [no]:
 > 


 Do you confirm generation? (yes/no) [yes]:
 > 

Generated or updated files
 Site path: /var/www/html
 1 - modules/custom/myroute/src/Controller/GreetingController.php
 2 - modules/custom/myroute/myroute.routing.yml
 // router:rebuild

 Rebuilding routes, wait a moment please


 [OK] Done rebuilding route(s).

Edit the greeting controller function to print the name.

public function greeting($name) {
  return [
    '#type' => 'markup',
    '#markup' => $this->t('Hello, @name!', ['@name' => $name]),
  ];
}

Try /hello/Joe.

/hello/123-456 works too. Let’s tone it down a little and add a validation criteria for the parameter.

Names can only have alphabets and spaces.

myroute.greeting_controller_greeting:
  path: 'hello/{name}'
  defaults:
    _controller: '\Drupal\myroute\Controller\GreetingController::greeting'
    _title: 'Greeting'
  requirements:
    _permission: 'access content'
    name: '[a-zA-z ]+'

Rebuild the cache for this to take effect.

$ drush cr

/hello/123-456 will now give a 404. Try /hello/Joe%20Pesci.

Custom permissions

How about adding custom permissions? What if we want to show the /hello page only to users who can administer content.

$ cd myroute
$ git checkout -f custom-access-check

We first indicate that the route uses custom permissions.

myroute.hello_world_controller_hello:
  path: 'hello'
  defaults:
    _controller: '\Drupal\myroute\Controller\HelloWorldController::hello'
    _title: 'Hello World'
  requirements:
    _custom_access: '\Drupal\myroute\Controller\HelloWorldController::custom_access_check'

Note that the _permission parameter has been replaced by _custom_access parameter.

Lets implement this custom access method inside the HelloWorldController.

// add the required namespaces at the top.
use Drupal\Core\Session\AccountInterface;
use Drupal\Core\Access\AccessResult;

  /**
   * Custom access check
   *
   * @param \Drupal\Core\Session\AccountInterface $account
   *   access checking done against this account.
   */
  public function custom_access_check(AccountInterface $account) {
    return AccessResult::allowedIf($account->hasPermission('access content') &&
      $account->hasPermission('administer content'));
  }

Rebuild cache and try hitting /hello as an anon user, you should get an “Access denied” page. Of note here is the AccessResult class, which was introduced to make access check related data cacheable.

Dynamic routes

Defining routes via a YAML file applies to static paths. If we want to route programmatically, we have to define and return a \Symfony\Component\Routing\Route object.

$ cd myroute
$ git checkout -f dynamic-routes

To illustrate the concept of dynamic routes, let’s take an imaginary requirement where we have a custom path, /content_types/{content_type}, which will display all the fields of a content type {content_type}. In order to create a dynamic route for the same, we have to create our own Routing class inside the src/Routing directory and override the routes() method. This is equivalent to its YAML cousin, the myroute.routing.yml file, but written in PHP.

<?php

/**
 * @file
 * Contains \Drupal\myroute\Routing\CTRoutes.
 */

namespace Drupal\myroute\Routing;

use Symfony\Component\Routing\Route;

/**
 * Dynamic routes for content types.
 */
class CTRoutes {

  /**
   * {@inheritdoc}
   */
  public function routes() {
    $routes = [];
    $content_types = \Drupal::service('entity.manager')->getStorage('node_type')->loadMultiple();
    foreach ($content_types as $content_type) {
      $routes['myroute.content_type_controller.' . $content_type->id() ] = new Route(
        '/content_types/' . $content_type->id(),
        array(
          '_controller' => '\Drupal\myroute\Controller\CTController::fields',
          '_title' => 'Field info for ' . $content_type->label(),
          'content_type' => $content_type->id(),
        ),
        array(
          '_permission'  => 'access content',
        )
      );
    }
    return $routes;
  }
}

The custom router above creates an array of routes for every content type in the system and returns it. For instance, if there are 3 content types in the system, like page, foo and bar, there will be 3 routes with paths /content_types/page/content_types/foo and /content_types/barrespectively.

Let’s flesh out the CTController as well.

<?php

namespace Drupal\myroute\Controller;

use Drupal\Core\Controller\ControllerBase;

/**
 * Class CTController.
 *
 * @package Drupal\myroute\Controller
 */
class CTController extends ControllerBase {

  /**
   * List fields info of a content type.
   *
   * @return string
   *   Return field list.
   */
  public function fields($content_type) {
    $render = '<table><tr><th>' . $this->t('Field type') . '</th><th>' . $this->t('Label') . '</th></tr>';
    $field_definitions = \Drupal::entityManager()->getFieldDefinitions('node', $content_type);
    foreach ($field_definitions as $field_name => $field_definition) {
      if (!empty($field_definition->getTargetBundle())) {
        $render .= '<tr><td>' . $field_definition->getType() . '</td><td>' . $field_definition->getLabel() . '</td></tr>';
      }
    }
    $render .= '</table>';
    return [
      '#type' => 'markup',
      '#markup' => $render,
    ];
  }
}

The field() method looks up the field definitions of the contenttype argument and renders it as a HTML table.

Finally, we have to indicate Drupal to pick up the custom Routing class we’ve added. This can be done by adding a _route_callbacks item to the routing YAML file.

route_callbacks:
  - '\Drupal\myroute\Routing\CTRoutes::routes'

Rebuild the cache and you are all set. Hit the content_types/article page, you should see something like this(provided you have the article CT in your system).

Tagged services in Drupal 8

We saw how to write a simple service container in Drupal earlier. We shall build a tagged service now. To illustrate a proper use case for tagged services, let’s contrive a scenario where you add a pipeline custom filters to user text before rendering it on the page.

First, clone the code which will be used in this post.

$ git clone git@github.com:drupal8book/process_text.git

Checkout the first version of the code where we take custom text from user, process and display it in a page without using services.

$ cd process_text
$ git checkout -f just-filtering

We get custom text from a user using a config form.

class CustomTextSettingForm extends ConfigFormBase {
  /**
   * {@inheritdoc}
   */
  protected function getEditableConfigNames() {
    return [
      'process_text.settings',
    ];
  }
  /**
   * {@inheritdoc}
   */
  public function getFormId() {
    return 'custom_text_setting_form';
  }
  /**
   * {@inheritdoc}
   */
  public function buildForm(array $form, FormStateInterface $form_state) {
    $config = $this->config('process_text.settings');
    $form['custom_text'] = [
      '#type' => 'textarea',
      '#title' => $this->t('Custom Text'),
      '#default_value' => $config->get('custom_text'),
    ];
    return parent::buildForm($form, $form_state);
  }
  /**
   * {@inheritdoc}
   */
  public function validateForm(array &$form, FormStateInterface $form_state) {
    parent::validateForm($form, $form_state);
  }
  /**
   * {@inheritdoc}
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {
    parent::submitForm($form, $form_state);
    $this->config('process_text.settings')
      ->set('custom_text', $form_state->getValue('custom_text'))
      ->save();
    $form_state->setRedirect('process_text.show');
  }
}

We save it as a piece of configuration called process_text.settings.custom_text.

Before rendering this text, let’s say you would want to:

  • Remove any <div> tags.
  • Substitute a token [greeting] with <span class"greeting">hello world</span>throughout the text.

We get the text and do all the above processing inside a custom controller.

class ProcessTextController extends ControllerBase {
  /**
   * Processtext.
   *
   * @return string
   *   Return processed custom text.
   */
  public function processText() {
    $custom_text = \Drupal::config('process_text.settings')->get('custom_text');
    // do processing
    // remove divs
    $custom_text = str_replace(["<div>", "</div>"], "", $custom_text);
    // replace greeting tokens
    $custom_text = str_replace("[greeting]", '<span class="greeting">hello world</span>', $custom_text);
    return [
      '#type' => 'markup',
      '#markup' => $custom_text
    ];
  }
}

This is good, but we could do better. What if we change the filter applying mechanism? We have to change this code. Instead, let’s convert it into a service.

$ cd process_text
$ git checkout -f services-first-cut

Our text filter service takes a set of filters and applies them to a given text when we call applyFilters.

class TextFilterService {
  private $filters = [];
  /**
   * @param Filter $filter
   */
  public function addFilter(Filter $filter) {
     $this->filters[] = $filter;
  }
    /**
     * applies all filters to given text and returns
     * filtered text.
     *
     * @param string $txt
     *
     * @return string
     */
  public function applyFilters($txt) {
    foreach ($this->filters as $filter) {
      $txt = $filter->apply_filter($txt);
    }
    return $txt;
  }
}

We need to crate a services.yml file for the above service.

services:
  process_text.text_filter:
    class: Drupal\process_text\TextFilterService

Here’s how the processText function text looks now.

public function processText() {
  $custom_text = \Drupal::config('process_text.settings')->get('custom_text');
  // do processing using a service
  $filter_service = \Drupal::service('process_text.text_filter');
  // remove divs
  $filter_service->addFilter(new RemoveDivs());
  // substitute greeting token
  $filter_service->addFilter(new Greeting());
  // apply all the above filters
  $custom_text = $filter_service->applyFilters($custom_text);
  return [
    '#type' => 'markup',
    '#markup' => $custom_text
  ];
}

Now the filter applying mechanism is swappable. We can add write a different functionality and inject that implementation using service containers.

Now, what if we want to add a new filter to this code, like, enclosing the whole text within a <p> tag.

Sure. We could do that.

Let’s checkout the specific tag where we add a new filter.

$ cd process_text
$ git checkout -f add-new-filter

We build that filter.

class EnclosePTags implements Filter {
  public function apply_filter($txt) {
    return '<p>'. $txt . '</p>';
  }
}

…and add it to the set of filters being applied.

public function processText() {
  $custom_text = \Drupal::config('process_text.settings')->get('custom_text');
  // do processing using a service
  $filter_service = \Drupal::service('process_text.text_filter');
  // remove divs
  $filter_service->addFilter(new RemoveDivs());
  // substitute greeting token
  $filter_service->addFilter(new Greeting());
  // Enclose p tags
  $filter_service->addFilter(new EnclosePTags());
  // apply all the above filters
  $custom_text = $filter_service->applyFilters($custom_text);
  return [
    '#type' => 'markup',
    '#markup' => $custom_text
  ];
}

How about injecting the filter adding mechanism itself? Wouldn’t it be cool if we are able to add new filters without changing this part of the code? Not to mention the fact that the code will be more testable than before if we follow this approach. This is exactly what tagged services help us accomplish.

Let’s write each filter as a tagged service.

$ cd process_text
$ git checkout -f tagged-services

Here’s how our process_text.services.yml looks now.

services:
  process_text.text_filter:
    class: Drupal\process_text\TextFilterService
    tags:
      - { name: service_collector, tag: text_filter, call: addFilter }

  remove_divs_filter:
    class: Drupal\process_text\TextFilter\RemoveDivs
    tags:
      - { name: text_filter }

  greeting_filter:
    class: Drupal\process_text\TextFilter\Greeting
    tags:
      - { name: text_filter }

  enclose_p_filter:
    class: Drupal\process_text\TextFilter\EnclosePTags
    tags:
      - { name: text_filter }

There are many changes here. Firstly, all the filters have been converted to services themselves. The have a common tag called text_filter. The main service also has a few changes. It has a tag called service_collector and a tag parameter call. This ritual of creating a service container and adding a set of tagged services is such a common pattern that Drupal 8 has a special tag to do this, called the service_collector. This tag takes an additional parameter called call which indicates what function has to be called in the service to add all the tagged services.

What happens is, Drupal’s TaggedHandlersPass picks up all services with “service_collector” tag, finds services which have the same tag as that of this service(text_filter in our case) and calls the method in call to consume the tagged service definition. If you’re coming from Symfony world, this might seem familiar for you. In order to execute some custom code, like applying a set of filters, we implement CompilerPassInterface, which is run whenever the service cotainer(ApplyFilter in our case) is being built. You can find more about CompilerPassInterface here.

Your controller code looks a lot simpler now.

public function processText() {
  $custom_text = \Drupal::config('process_text.settings')->get('custom_text');
  // do processing using a service
  $filter_service = \Drupal::service('process_text.text_filter');
  $custom_text = $filter_service->applyFilters($custom_text);
  return [
    '#type' => 'markup',
    '#markup' => $custom_text
  ];
}

Now, all you need to add new filters is to update the service yaml file with the new filter service and tag it with “text_filter” tag.

Tagged services in the wild

Drupal allows developers to add a new authentication mechanism using tagged services. The authentication_collector is defined in core.services.yml.

authentication_collector:
  class: Drupal\Core\Authentication\AuthenticationCollector
  tags:
    - { name: service_collector, tag: authentication_provider, call: addProvider }

To add a new authentication provider, one has to implement the AuthenticationProviderInterfaceand flesh out the applies and authenticate functions. This will be the subject of another post.

Here’s how the addProvider function looks like:

public function addProvider(AuthenticationProviderInterface $provider, $provider_id, $priority = 0, $global = FALSE) {
  $this->providers[$provider_id] = $provider;
  $this->providerOrders[$priority][$provider_id] = $provider;
  // Force the providers to be re-sorted.
  $this->sortedProviders = NULL;

  if ($global) {
    $this->globalProviders[$provider_id] = TRUE;
  }
}

And here’s how we would register our hypothetical authentication provider service.

services:
  authentication.custom_auth:
    class: Drupal\custom_auth\Authentication\Provider\CustomAuth
    tags:
      - { name: authentication_provider }

Another example is the breadcrumb manager service.

breadcrumb:
  class: Drupal\Core\Breadcrumb\BreadcrumbManager
  arguments: ['@module_handler']
  tags:
    - { name: service_collector, tag: breadcrumb_builder, call: addBuilder }

To add breadcrumbs from your module, you would need to implement BreadcrumbBuilderInterface and add the following entry to your services.yml,

services:
  foo.breadcrumb:
    class: Drupal\foo\MyBreadcrumbBuilder
    tags:
      - { name: breadcrumb_builder, priority: 100 }

The BreadcrumbManager::addBuilder collects all breadcrumb bilders and builds it using the BreadcrumbManager::build function.

public function addBuilder(BreadcrumbBuilderInterface $builder, $priority) {
  $this->builders[$priority][] = $builder;
  // Force the builders to be re-sorted.
  $this->sortedBuilders = NULL;
}

Drupal composer workflow – part 2

In the previous post, we saw how to add and manage modules and module dependencies in Drupal 8 using Composer.

In this post we shall see how to use an exclusive composer based Drupal 8 workflow. Let’s start with a vanilla Drupal install. The recommended way to go about it is to use Drupal Composer project.

$ composer create-project drupal-composer/drupal-project:8.x-dev drupal-8.dev

If you are a careful observer(unlike me), you will notice that a downloaded Drupal 8 package ships with the vendor/ directory. In other words, we need not install the composer dependencies when we download it from d.o. On the other hand, if you “git cloned” Drupal 8, it won’t contain the vendor/ directory, hence the extra step to run `composer install` in root directory. The top level directory contains a composer.json and the name of the package is drupal/drupal, which is more of a wrapper for the drupal/core package inside the core/ directory. The drupal/core package installs Drupal core and its dependencies. The drupal/drupal helps you build a site around Drupal core, maintains dependencies related to your site and modules etc.

Drupal project takes a slightly different project structure.It installs core and its dependencies similar to drupal/drupal. It also installs the latest stable versions of drush and drupal console.

$ composer create-project drupal-composer/drupal-project:8.x-dev d8dev --stability dev --no-interaction

New directory structure

Everything Drupal related goes in the web/ directory, including coremodulesprofiles and themes. Contrast this with the usual structure where there is a set of top level directories named coremodulesprofiles and themes.

drush and drupal console(both latest stable versions) gets installed inside vendor/bindirectory.The reason Drush and Drupal console are packaged on a per project basis is to avoid any dependency issues which we might normally face if they are installed globally.

How to install Drupal

Drupal can be installed using the typical site-install command provided by drush.

$ cd d8dev/web
$ ../vendor/bin/drush site-install --db-url=mysql://<db-user-name>:<db-password>@localhost/<db-name> -y

Downloading modules

Modules can be downloaded using composer. They get downloaded in the web/modules/contribdirectory.

$ cd d8dev
$ composer require drupal/devel:8.1.x-dev

The following things happen when we download a module via composer.

  1. Composer updates the top level composer.json and adds drupal/devel:8.1.x-dev as a dependency.
    "require": {
        "composer/installers": "^1.0.20",
        "drupal-composer/drupal-scaffold": "^2.0.1",
        "cweagans/composer-patches": "~1.0",
        "drupal/core": "~8.0",
        "drush/drush": "~8.0",
        "drupal/console": "~1.0",
        "drupal/devel": "8.1.x-dev"
    },
  1. Composer dependencies(if any) for that module get downloaded in the top level vendor directory. These are specified in the composer.json file of that module. At the time of writing this, Devel module does not have any composer dependencies.
     "license": "GPL-2.0+",
      "minimum-stability": "dev",
      "require": { }
    }

Most modules in Drupal 8 were(are) written without taking composer into consideration. We use the drush dl command every time which parses our request and downloads the appropriate version of the module from drupal.org servers. Downloading a module via composer requires the module to have a composer.json as a minimal requirement. So how does composer download all Drupal contrib modules if they don’t have any composer.json? The answer lies in a not so secret sauce ingredient we added in our top level composer.json:

"repositories": [
    {
        "type": "composer",
        "url": "https://packagist.drupal-composer.org"
    }
],

Composer downloads all packages from a central repository called Packagist. It is the npmjsequivalent of PHP. Drupal provides its own flavour of Packagist to serve modules and themes exclusively hosted at Drupal.org. Drupal packagist ensures that contrib maintainers need not add composer.json to their project.

Let’s take another module which does not have a composer.json, like Flag(at the time of writing this). Let’s try and download flag using composer.

$ composer require drupal/flag:8.4.x-dev
./composer.json has been updated
> DrupalProject\composer\ScriptHandler::checkComposerVersion
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing drupal/flag (dev-8.x-4.x 16657d8)
    Cloning 16657d8f84b9c87144615e4fbe551ad9a893ad75

Writing lock file
Generating autoload files
> DrupalProject\composer\ScriptHandler::createRequiredFiles

Neat. Drupal Packagist parses contrib modules and serves the one which matches the name and version we gave when we ran that “composer require” command.

Specifying package sources

There is one other step you need to do to complete your composer workflow, i.e., switching to the official Drupal.org composer repository. The actual composer.json contains Drupal packagist as the default repository.

"repositories": [
    {
        "type": "composer",
        "url": "https://packagist.drupal-composer.org"
    }
],

Add the Drupal.org composer repo using the following command:

$ composer config repositories.drupal composer https://packages.drupal.org/8

Now, your repositories entry in composer.json should look like this:

"repositories": {
       "0": {
           "type": "composer",
           "url": "https://packagist.drupal-composer.org"
       },
       "drupal": {
           "type": "composer",
           "url": "https://packages.drupal.org/8"
       }
   }

To ensure that composer indeed downloads from the new repo we specified above, let’s remove the drupal packagist entry from composer.json.

$ composer config --unset repositories.0

The repositories config looks like this now:

"repositories": {
      "drupal": {
          "type": "composer",
           "url": "https://packages.drupal.org/8"
      }
  }

Now, let’s download a module from the new repo.

$ composer require drupal/token -vvv

As a part of the verbose output, it prints the following:

...
Loading composer repositories with package information
Downloading https://packages.drupal.org/8/packages.json
Writing /home/lakshmi/.composer/cache/repo/https---packages.drupal.org-8/packages.json into cache
...

which confirms that we downloaded from the official package repo.

Custom package sources

Sometimes, you might want to specify your own package source for a custom module you own, say, in Github. This follows the usual conventions for adding VCS package sources in Composer, but I’ll show how to do it in Drupal context.

First, add your github URL as a VCS repository using the composer config command.

$ composer config repositories.restful vcs "https://github.com/RESTful-Drupal/restful"

Your composer.json will look like this after the above command is run successfully:

"repositories": {
    "drupal": {
        "type": "composer",
        "url": "https://packages.drupal.org/8"
    },
    "restful": {
        "type": "vcs",
        "url": "https://github.com/RESTful-Drupal/restful"
    }
}

If you want to download a package from your custom source, you might want it to take precedence to the official package repository, as order really matters for composer. I haven’t found a way to do this via cli, but you can edit the composer.json file and swap both package sources to look like this:

"repositories": {
    "restful": {
        "type": "vcs",
        "url": "https://github.com/RESTful-Drupal/restful"
    },
    "drupal": {
        "type": "composer",
        "url": "https://packages.drupal.org/8"
    }
}

Now, lets pick up restful 8.x-3.x. We can specify a Github branch by prefixing with a “dev-“.

$ composer require "drupal/restful:dev-8.x-3.x-not-ready"

Once restful is downloaded, composer.json is updated accordingly.

"require": {
     "composer/installers": "^1.0.20",
     "drupal-composer/drupal-scaffold": "^2.0.1",
     "cweagans/composer-patches": "~1.0",
     "drupal/core": "~8.0",
     "drush/drush": "~8.0",
     "drupal/console": "~1.0",
     "drupal/devel": "8.1.x-dev",
     "drupal/flag": "8.4.x-dev",
     "drupal/mailchimp": "8.1.2",
     "drupal/token": "1.x-dev",
     "drupal/restful": "dev-8.x-3.x-not-ready"
 },

Updating drupal core

Drupal core can be updated by running:

$ composer update drupal/core
> DrupalProject\composer\ScriptHandler::checkComposerVersion
Loading composer repositories with package information
Updating dependencies (including require-dev)                                         
  - Removing drupal/core (8.1.7)
  - Installing drupal/core (8.1.8)
    Downloading: 100%         

Writing lock file
Generating autoload files
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
    Downloading: 100%         
> DrupalProject\composer\ScriptHandler::createRequiredFiles

As the output reads, we updated core from 8.1.7 to 8.1.8. We will revisit the “Writing lock file” part in a moment. After this step is successful, we have to run drush updatedb to do any database updates. This applies to even updating modules.

$ cd d8dev/web
$ ../vendor/bin/drush updatedb

Updating modules

Updating modules in composer workflow is can be done by the composer update command. For updating a module, say, devel we would do:

$ composer update drupal/devel
> DrupalProject\composer\ScriptHandler::checkComposerVersion
Loading composer repositories with package information
Updating dependencies (including require-dev)                                         
Nothing to install or update
Generating autoload files
> DrupalProject\composer\ScriptHandler::createRequiredFiles

Hmmm. Looks like devel is already the latest bleeding edge version. To quickly revise and ensure what composer related artifacts we need to check in to version control,

Should you check in the vendor/ directory?

Composer recommends that you shouldn’t, but there are some environments that don’t support composer(ex. Acquia Cloud), in which case you have to check in your vendor folder too.

Should you check in the composer.json file?

By now, you should know the answer to this question 🙂

Should you check in the composer.lock file?

Damn yes.

composer.lock contains the exact version of the dependencies which are installed. For example, if your project depends on Acme 1.*, and you install 1.1.2 and your co-worker runs composer install after a month or so, it might install Acme 1.1.10, which might introduce version discrepancies in your project. To prevent this, composer install will check if a lock file exists, and install only that specific version recorded or “locked” down in the lock file. The only time the lock file changes is when you run a composer update to update your project dependencies to their latest versions. When that happens, composer updates the lock file with the newer version that got installed.

Drinking the Drupal-Composer Kool-Aid?

Do you want to manage modules and dependencies the PHP way instead of the “Drupal” way? Don’t know how to use composer with Drupal? Are you planning to ditch drush make approach and adopt a composer based workflow?

If you answered yes to any of the above questions, then you should read this post.

Compose what?

Composer is PHP equivalent of Python’s pip, Ruby’s bundler and Node’s npm. It helps manage projects and dependencies for a PHP project.

Why Composer?

The average PHP developer wrote 3.5 frameworks in their lifetime. This was before the era of package management. Then came along PEAR. PEAR did the job, but had some flaws. To point out one, for instance, PEAR installs packages globally instead of installing them on a per-project basis. So, if you install Acme libary v1.2, you had to use Acme v1.2 throughout all of your projects. Thankfully, Composer has succeeded PEAR as the de facto package manager for PHP.

The Drupal context

Modules and themes in Drupal are managed by drush, specifically drush make. Both composer and drush make do the same thing, i.e. download and install specific versions of Drupal modules and themes. Composer has an extra ingredient called autoload. This automagically figures out where your dependency files are there and includes them in your code as needed. Drupal 8 core adopted composer as the dependency manager. So if composer is great, why isn’t everybody using it? To bridge this gap, Drupal 8 brought in a module called composer manager. This module helps manage composer dependencies of contrib modules. This might otherwise be done by editing Drupal 8 core’s root composer.json and adding our module’s composer dependencies there(which amounts to hacking the core).

So, if module acme depends on package foo, it will have a composer.json in the module’s root directory with foo added as a dependency. Composer manager will add foo to Drupal’s vendor directory. Starting with Drupal 8.1, we no longer need Composer manager to manage module dependencies. It can be done by composer alone. Let’s take a look at how we pull this off.

Make sure you have Drupal 8.1.x installed and setup. Let’s download the address module using composer.

/v/w/h/drupal-8.1.1↪ composer require drupal/address "8.1.*@dev
Warning: This development build of composer is over 60 days old. It is recommended to update it by running "/usr/local/bin/composer self-update" to get the latest version.
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - The requested package drupal/address could not be found in any version, there may be a typo in the package name.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion> for more details.

Read <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.

Installation failed, reverting ./composer.json to its original content.

Add packagist as a repository in your composer.json for the lookup to succeed.

NOTE this is a one time thing and you needn’t do it for every module.

/v/w/h/drupal-8.1.1↪ composer config repositories.drupal composer https://packagist.drupal-composer.org/

Your `composer.json` will have an entry for `repositories.drupal`.

"repositories": {
    "drupal": {
        "type": "composer",
        "url": "https://packagist.drupal-composer.org/"
    }
}

Now, another shot at it.

/v/w/h/drupal-8.1.1↪ composer require drupal/address "8.1.*@dev" 
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Nothing to install or update
Generating autoload files
> Drupal\Core\Composer\Composer::preAutoloadDump
> Drupal\Core\Composer\Composer::ensureHtaccess

NOTE that this might take a bit of time if you have a lot of composer dependencies cached.

Enable address module, either via the UI or using drush.

/v/w/h/drupal-8.1.1↪ drush en address -y                                                                                  
The following extensions will be enabled: address
Do you really want to continue? (y/n): y
address was enabled successfully.                                                                                                          [ok]
address defines the following permissions: administer address formats, administer zones

All set to rock!

So, what happened behind the scenes?

Address module had a composer.json which reads like this:

{
  "name": "drupal/address",
  "type": "drupal-module",
  "description": "Provides functionality for storing, validating and displaying international postal addresses.",
  "homepage": "http://drupal.org/project/address",
  "license": "GPL-2.0+",
  "require": {
    "commerceguys/intl": "dev-master",
    "commerceguys/addressing": "dev-master",
    "commerceguys/zone": "dev-master"
  },
  "minimum-stability": "dev"
}

The above commerceguys/* dependencies got installed in the main vendor/ directory by composer.

Composer vendor directory
Composer vendor directory

What are the shortcomings of using composer alone to manage modules? For one, it would be too developer focused, in the sense that site builders(common audience for Drupal) won’t be able to use it because of the complexity involved. When they have to download a module and enable it(a very common operation in Drupal), they can’t do it entirely through the UI and have to run composer commands in the backend.

How to update modules using composer? We shall talk about that in the next post.

Publishing a book using org-mode

I’ve been flirting with org-mode lately and writing a book at Leanpub. Why not use org mode to do it, I thought, and this is the result. There is already an awesome org-mode exporter for leanpub. Go ahead and download it. This post builds upon it.

Before we dive into the org-mode side of things, let’s look at how Leanpub publishing works.

Leanpub follows the KISS principle to write books. You can use Github or Dropbox to sync your book. We will be using Dropbox in this post. When you opt to create a book using the Dropbox sync method, you will get a sharing invite to your Dropbox account from “Leanpub Bookbot”. This will create 2 folders called manuscript and convert_html. The manuscript is your sandbox directory where you add your book chapters in markdown(or markua). The chapters which need to be in the book will be added to a Book.txt file in the same directory and the chapters that form a part of your book sample will be put up in a Sample.txt file. More info can be found here.

But why org-mode?

Firstly, because I find it super convenient to edit all the chapters and move them around in a single file. This works great in org-mode if each chapter is a top level heading in an org file. Also, I can add a lot of metadata about a section/chapter as notes. These are things to keep track, but not meant to be published. For example, I use an org drawer to store the last published date of the book, and I clock the time spent in writing each chapter. That’s something org-mode helps you do seamlessly. In fact, I store my entire work log of the book in the book’s org file. The best part is, I can do version control of the book(a single org file) using git. Org-mode also has this ability to add code blocks, which can be executed and the results be stored alongside the code blocks in the same file(though I don’t use the results feature for my book). That’s a LOT of excuses to use org-mode to write your next Leanpub book(or any other book). Lastly, you can export your .org file to loads of formats. There’s one plugin which exports to twitter bootstrap flavoured HTML as well!

How it works

The underlying idea is to map each top level heading in an org-file to a chapter in your Leanpub book. If you want to exclude a specific section(called a sub-tree in org-mode lingo) or an entire chapter from getting exported into the book, just add the “noexport” tag to that heading.

Org helps you set certain file-level properties, like indentation, collapsing and unfolding of parts of text in your buffer and generating table of contents. This can be set by adding:

#+STARTUP: indent showeverything

at the beginning of the file.

It’s useful to preset the tags your org-file can contain using the TAGS property.

#+TAGS: noexport sample

We don’t want org-mode to generate the table of contents for us, it will be automatically done by Leanpub. So, we quieten that setting.

#+OPTIONS: toc:nil

You can set custom workflow states to each heading using the following property:

#+TODO: TODO(t) DRAFT(f@/!) IN-THE-BOOK(i!) | DONE(d!) CANCELED(c)

The states to the left of the “|” indicates that the task in in some form of progress and the ones to the right of ‘|’ indicates some form of closure of the item. The ‘!’ implies that when the workflow state changes, a timestamp will be recorded beneath the entry. The ‘@’ indicates that the timestamp will be recorded along with a note when there is a workflow change. When I was initially fiddling with it, this metadata was also getting exported along with the actual book contents.

For example, here’s how a typical chapter looked in my org file:

* DRAFT Routing and controllers                           :sample:
- State "DRAFT"      from "30%"        [2016-05-30 Mon 21:08]
- State "30%"        from "TODO"       [2016-05-26 Thu 17:05]

Routing is responsible for matching a URL path with a custom content or functionality in your site.

To avoid this, I needed to add another property called logdrawer,

#+STARTUP: indent showeverything logdrawer

so that state changes are logged under a property drawer called LOGBOOK.

* DRAFT Routing and controllers                           :sample:
:LOGBOOK:
- State "DRAFT"      from "30%"        [2016-05-30 Mon 21:08]
- State "30%"        from "TODO"       [2016-05-26 Thu 17:05]
:END:

Routing is responsible for matching a URL path with a custom content or functionality in your site.

The filename of each exported top-level heading can be specified by an EXPORT_FILE_NAMEproperty, as in:

* Drupal permissions and users
:PROPERTIES:
:EXPORT_FILE_NAME: permissions-and-users.txt
:END:

Chapters to be included as part of the sample book should have the “sample” tag.

Here’s the complete function to export an org-buffer into a Leanpub book.

(defun leanpub-export ()
  "Export buffer to a Leanpub book."
  (interactive)
  (if (file-exists-p "./Book.txt")
  (delete-file "./Book.txt"))
  (if (file-exists-p "./Sample.txt")
  (delete-file "./Sample.txt"))
  (org-map-entries
   (lambda ()
     (let* ((level (nth 1 (org-heading-components)))
            (tags (org-get-tags))
           (title (or (nth 4 (org-heading-components)) ""))
           (book-slug (org-entry-get (point) "TITLE"))
           (filename
            (or (org-entry-get (point) "EXPORT_FILE_NAME") (concat (replace-regexp-in-string " " "-" (downcase title)) ".md"))))
       (when (= level 1) ;; export only first level entries
         ;; add to Sample book if "sample" tag is found.
         (when (or (member "sample" tags) (string-prefix-p "frontmatter" filename) (string-prefix-p "mainmatter" filename))
           (append-to-file (concat filename "\n\n") nil "./Sample.txt"))
         (append-to-file (concat filename "\n\n") nil "./Book.txt")
         ;; set filename only if the property is missing
         (or (org-entry-get (point) "EXPORT_FILE_NAME")  (org-entry-put (point) "EXPORT_FILE_NAME" filename))
         (org-leanpub-export-to-markdown nil 1 nil)))) "-noexport") (org-save-all-org-buffers)
   nil nil)

NOTE you should have org-leanpub exporter installed to run this function.

Let’s dissect this function a bit. The main API called here is org-map-entries, which maps every org element in the buffer to a function. This function checks if the element is a top level element, in which case it calls org-leanpub exporter for that sub-tree. org-map-entries accepts an optional match parameter. In our case, we want to apply the function only if it does not have the “noexport” tag, indicated by a -noexport argument.

Leanpub requires a set of special {mainmatter}{frontmatter} and {backmatter} files to indicate various portions of the book, like Appendix, for example. This is indicated by the following org-mode headlines in appropriate places in your file.

* Frontmatter
:PROPERTIES:
:EXPORT_FILE_NAME: frontmatter.md
:END:
{frontmatter}

* Mainmatter
:PROPERTIES:
:EXPORT_FILE_NAME: mainmatter.md
:END:
{mainmatter}

* Backmatter
:PROPERTIES:
:EXPORT_FILE_NAME: backmatter.md
:END:
{backmatter}

Bonus – generate your book’s preview from emacs

Leanpub has an API endpoint for generating your book preview, which means you can issue a POST call to Leanpub to trigger book generation for preview. To do this,

  • You need to generate an API key. The Leanpub site has instructions on how to do this.
  • Install the emacs request library to issue API requests.

Here’s the preview generation function:

(defun leanpub-preview ()
  "Generate a preview of your book @ Leanpub."
  (interactive)
  (request
   "https://leanpub.com/<YOUR-BOOK-SLUG>/preview.json" ;; or better yet, get the book slug from the buffer
   :type "POST"                                        ;; and construct the URL
   :data '(("api_key" . "53cr3t"))
   :parser 'json-read
   :success (function*
             (lambda (&key data &allow-other-keys)
               (message "Preview generation queued at leanpub.com."))))
  )

Have fun writing your next book entirely in org-mode!

Writing your first Drupal service

We got a basic understanding of what a Drupal service is previously. Let’s put that into action by creating a simple service in Drupal 8. Take note that this service what we are about to create doesn’t do anything useful yet, and is written with a pedagogical purpose in mind.

This service will list a few best selling books from Leanpub. First, to deal with all the boilerplate of creating a module using DrupalConsoleNever ever write a module from scratch by hand!

$ drupal generate:module

 // Welcome to the Drupal module generator

 Enter the new module name:
 > techbooks

 Enter the module machine name [techbooks]:
 > 

 Enter the module Path [/modules/custom]:
 > 

 Enter module description [My Awesome Module]:
 > My first service

 Enter package name [Custom]:
 > 

 Enter Drupal Core version [8.x]:
 > 

 Do you want to generate a .module file (yes/no) [no]:
 > no

 Define module as feature (yes/no) [no]:
 > no

 Do you want to add a composer.json file to your module (yes/no) [yes]:
 > no

 Would you like to add module dependencies (yes/no) [no]:
 > no


 Do you confirm generation? (yes/no) [yes]:
 > yes

Generated or updated files
 Site path: /var/www/html/drupal-8.0.2
 1 - modules/custom/techbooks/techbooks.info.yml

The next step is to write a service configuration in our module. This configuration dictates how the service class will be initialized and how it can be called from code. The Drupal convention is to write this in a <module-name>.services.yml.

services:
  techbooks.listbooks:
    class: Drupal\techbooks\ListBooks

The services configuration part begins with services: and lists all the services provided by our module. In our case, we have just one service. We give it a name, techbooks.listbooks. Its a good idea to namespace services, which can be done by prefixing a namespace name followed by a dot. The class tells Drupal what class to call to create the service. This should be the fully qualified name of the class. The services YAML file can also store any service related information as well. All the book related data like title, author etc. is stored in the same file.

parameters:
  leanpub.booklist:
    - {title: 'The Elements of Data Analytic Style', author: 'Jeff Leek', url: 'https://leanpub.com/datastyle'}
    - {title: "Build APIs You Won't Hate", author: 'Phil Sturgeon', url: 'https://leanpub.com/build-apis-you-wont-hate'}
    - {title: 'Easy Laravel 5', author: 'W. Jason Gilmore', url: 'https://leanpub.com/easylaravel'}
    - {title: 'Ansible for DevOps', author: 'Jeff Geerling', url: 'https://leanpub.com/ansible-for-devops'}
    - {title: 'Principles of Package Design', author: 'Matthias Noback', url: 'https://leanpub.com/principles-of-package-design'}
    - {title: 'Modernizing Legacy Applications In PHP', author: 'Paul M. Jones', url: 'https://leanpub.com/mlaphp'}
    - {title: 'Front-End Fundamentals', author: 'Carwin Young, Joe Fender', url: 'https://leanpub.com/front-end-fundamentals'}
    - {title: 'Talking with Tech Leads', author: 'Patrick Kua', url: 'https://leanpub.com/talking-with-tech-leads'}

We shall come back in a bit on how we will use this data. Note that creating a service is also mostly boilerplate code. Hence, we can avail Drupal Console to do the job for us and fill out the custom details.

$ drupal generate:service

 // Welcome to the Drupal service generator
 Enter the module name [techbooks]:
 > 

 Enter the service name [techbooks.default]:
 > techbooks.listbooks

 Enter the Class name [DefaultService]:
 > ListBooks

 Create an interface (yes/no) [yes]:
 > no


 Do you want to load services from the container (yes/no) [no]:
 > no


 Do you confirm generation? (yes/no) [yes]:
 > yes

Generated or updated files
 Site path: /var/www/html/drupal-8.0.2
 1 - modules/custom/techbooks/techbooks.services.yml
 2 - modules/custom/techbooks/src/ListBooks.php

 Rebuilding cache(s), wait a moment please.


 [OK] Done clearing cache(s).

We have defined a service, but haven’t exactly told Drupal what it does. For this, we have to flesh out the ListBooks class inside the module’s src/ directory.

namespace Drupal\techbooks;


/**
 * Class ListBooks.
 *
 * @package Drupal\techbooks
 */
class ListBooks {
  public function getBooks() {
    return t('All your books will be listed!');
  }
}

Once you have defined this service and enabled the module, you can use it anywhere in your code by invoking the service:

$book_list = \Drupal::service('techbooks.listbooks')->getBooks();

Drupal creates an instance of your listbooks service and fetches that instance on every call to the service. It effectively calls getBooks() of that instance, which returns a string. Let’s add a route for listing books.

If you hit /techbooks in your browser, you should get this:

Drupal 8 services

Not a lot happening there. Let’s actually list some books!

Services can take in parameters while initializing. They are just plain PHP classes in disguise after all! Let’s pass the book list we defined earlier in the yaml file as a parameter to the listbooksservice.

services:
  techbooks.listbooks:
    class: Drupal\techbooks\ListBooks
    arguments: ["%leanpub.booklist%"]

The only change is the arguments property. We refer to the leanpub.booklist property as the argument. The enclosing % is to indicate that it is an internal variable/property. Also, notice that we wrap the argument within quotes, failing which it will be interpreted as a literal string. For example, arguments: [leanpub.booklist] will send the string “leanpub.booklist” as argument to the ListBooks class, which is not what we want!

Let’s modify the ListBooks class to take in the arguments.

class ListBooks {

  public function __construct($books) {
    $this->books = $books;
  }

  public function getBooks() {
    return $this->books;
  }
}

Changing the controller code accordingly,

public function bookList() {
  $book_list = \Drupal::service('techbooks.listbooks')->getBooks();
  $render = '';
  foreach($book_list as $book) {
    $render .= '<div><span>Title: ' . $book['title'] . '</span></div>';
    $render .= '<div><span>Author: ' . $book['author'] . '</span></div>';
  }
  return [
      '#type' => 'markup',
      '#markup' => $render
  ];
}

Now, our /techbooks page actually lists books!

Drupal 8 functional service
Drupal 8 functional service

It doesn’t look pretty and all, but hey, we just created our first Drupal service 🙂

You can grab the code for this example from github and checkout the basic-service tag to see it come to life.

Resources

Understanding Drupal 8 services

Drupal 8 has this concept of services, or reusable PHP objects. The distinguishing feature of services is that their initiation is configurable. In other words, we can configure what type of instance is created and when this instance is created in the code, mostly without changing the code at all.

service

Image credits

Why is this a big deal? Assume that you are writing a custom Drupal module which handles a lot of transactional e-mail(a fancy term for e-mail sent to an individual based on some action, like if anyone replies to your comment, for instance). You decide to use some transactional e-mail service like Postmark or Mandrill.

The straightforward way to go about this, is to create a wrapper around the respective API offered by the service. What if we want to migrate to another transactional e-mail service, like Amazon SES? Typically, this would involve rewriting the wrapper around this new service and change the service instance to the newer one throughout the codebase. Services aim to solve this “code rewrite” problem. Let’s see how we would tackle this using a “Services” approach.

In the services way of doing things, you will expose the transactional e-mail wrapper as a service. This service would be initiated in your module and invoked whenever you want to send e-mail. The e-mail service you use to send mails, its API keys, configuration etc. will be service parameters. Migrating to a new e-mail service would be as simple as rewriting the service configuration. I wouldn’t say it is a zero code change, but its a vast improvement over the previous approach.

There are other benefits of using services. Your code becomes easily testable. Want to test your module’s e-mail functionality? Swap your service with a test e-mail service and you are done. Easy peasy!

But they seem very similar to PHP objects

Services sound a lot like PHP objects, except that we specify them via configuration, instead of code. But there’s more to it than meets the eye. Services are instantiated lazily, which means no service object is created if the service is not invoked. The other difference is, the same object is returned every time you invoke the service instead of creating new instances.

So what are plugins then?

Plugins are reusable objects discoverable by annotations. Though they are similar to services by definition, they differ in many ways. If you want to expose an interface which can exhibit different behaviour, go for a plugin.An example would be image styles. This is detailed here.

In the next post, we shall see how to create a simple Drupal service.