Tim’s Weblog Tim's Weblog
Tim Strehle’s links and thoughts on Web apps, managing software development and Digital Asset Management, since 2002.
2021-08-14

A Companion for WoodWing Assets

At the SPIEGEL-Verlag, we use the WoodWing Assets DAM software in production since January 2020. Most of the images you see on spiegel.de (one of Germany’s largest news sites) are sent to our Web CMS from Assets.

WoodWing Assets is a pretty good DAM product, focused on a relatively small set of core features. There is little built-in business logic or connectors – instead, Assets provides a couple of extension mechanisms so you can customize it: Action plugins and Panel plugins (custom HTML + JavaScript embedded into the Assets UI), a REST API, Webhooks, and API plugins (a wrapper for custom REST services).

We made use of action and panel plugins to add functionality to the Assets UI (like dialogs for “advanced search” and Web CMS export). But there is also a bunch of “invisible” logic. Here are a few of our use cases:

  • moving assets into the right Assets folder after upload into a collection
  • reading IPTC Credit and Creator fields (on import or update), and combining them in a “calculated credit” metadata field, ready to use in our CMS
  • updating (via the CMS API) the image credit shown on the spiegel.de Web site when the credit is changed within Assets
  • depublishing images from the Web site when the expiry date is reached
  • routing files through OneVision Amendo for automatic image optimization

Since we were dealing with multiple workflows and systems, we knew that simple point-to-point integrations would become difficult to coordinate and maintain. We decided to bundle all of that logic into a dedicated service, and developed a Web service application we call the “Companion”. Other systems are allowed to use the Assets REST API directly for read-only purposes, but all data modification and Assets Webhook handling is done exclusively within the Companion.

Diagram: Companion as a black box

The Companion is written in PHP and has no UI. Its main interfaces are a Webhook receiver and a small API. It builds upon the Assets client PHP library we open sourced, but the Companion itself is not open source (still, let us know if you’re interested in using it).

Diagram: What’s in the box

The Webhook receiver decides which actions should be triggered by an incoming Assets event (we can easily plug in code for new use cases using Symfony Event Subscribers). Any action involving another system is not executed immediately, but added to a RabbitMQ queue (or a Camunda instance for more complex workflows) and processed asynchronously by worker processes (using the Symfony Messenger). We use one queue and one set of worker processes per integrated system so we can scale processes individually, and stop them when that system has a downtime. 

We run the Companion in an OpenShift cluster (based on Docker / Kubernetes). Deployment and maintenance are super simple, and there even is a nice UI with 1-click scaling 😉

OpenShift Web Console

Want to see some code? This code snippet running in the Webhook receiver catches all “file created” events and adds a “check for duplicates” event to RabbitMQ:

<?php

class AssetLinkDuplicatesSubscriber extends AssetsWebhookSubscriber
{
    public static function getSubscribedEvents()
    {
        return [
            AssetCreateEvent::class => 'handle',
            AssetCreateByCopyEvent::class => 'handle',
            AssetCreateFromFilestoreRescueEvent::class => 'handle',
            AssetCreateFromVersionEvent::class => 'handle'
        ];
    }

    public function handle(AssetsWebhookEvent $assetsWebhookEvent): void
    {
        // Skip variations
        if (strlen($assetsWebhookEvent->getMasterId()) > 0) {
            return;
        }

        // ignore if this is a container (e.g. collection)
        if ($assetsWebhookEvent->getAssetDomain() === 'container') {
            return;
        }

        $this->messageBus->dispatch(new AssetLinkDuplicatesMessage($assetsWebhookEvent));
    }
}

The worker process runs this code to perform the actual work, searching Assets for identical files and creating relations between the assets:

<?php

class AssetLinkDuplicatesHandler implements MessageHandlerInterface
{
    const RELATION_TYPE = 'duplicate';

    protected AssetsClient $assetsClient;
    protected LoggerInterface $logger;

    public function __construct(LoggerInterface $logger, AssetsClient $assetsClient)
    {
        $this->assetsClient = $assetsClient;
        $this->logger = $logger;
    }

    public function __invoke(AssetLinkDuplicatesMessage $message): void
    {
        $assetsWebhookEvent = $message->getAssetsWebhookEvent();

        // Skip variations
        if (strlen($assetsWebhookEvent->getMasterId()) > 0) {
            return;
        }

        $asset = $this->assetsClient->searchAsset($assetsWebhookEvent->getAssetId());

        try {
            if (empty($asset->getMetadata()['firstExtractedChecksum'])) {
                throw new RuntimeException(sprintf('%s: Failed to find the firstExtractedChecksum for asset %s',
                    __METHOD__, $asset->getId()));
            }

            // check if there are other (original) assets with the same firstExtractedChecksum

            $query = sprintf(
                'firstExtractedChecksum:%s -id:%s -masterId:*',
                $asset->getMetadata()['firstExtractedChecksum'],
                $asset->getId()
            );

            $response = $this->assetsClient->search((new SearchRequest($this->assetsClient->getConfig()))
                ->setQ($query));

            if ($response->getTotalHits() === 0) {
                // no assets to relate
                return;
            }

            foreach ($response->getHits() as $hit) {
                $relationRequest = new CreateRelationRequest($this->assetsClient->getConfig());
                $relationRequest->setTarget1Id($asset->getId());
                $relationRequest->setTarget2Id($hit->getId());
                $relationRequest->setRelationType(self::RELATION_TYPE);
                $this->assetsClient->createRelation($relationRequest);
            }

        } catch (RuntimeException $e) {
            throw new RuntimeException(sprintf('%s: Failed to create duplicate relations for asset %s %s', __METHOD__,
                $asset->getId(), $e->getMessage()), $e->getCode(), $e);
        }
    }
}

The Companion has been running in production since our Assets launch one and a half years ago, and so far it has been proven to be fast, very stable (the downtimes we had were related to the OpenShift environment), and easily adapted to new requirements. We’re very happy with it 🙂

Sat, 14 Aug 2021 22:13:00 +0000