Globally scoped platform ARG's in Docker BuildKit

ARG a.k.a “build-time variable” can be used to pass a variable to builder with the docker build command using --build-arg <varname>=<value> flag

With the new engine 18.09 Docker supports a new backend BuildKit for executing your builds.

You may switch to this backend by setting the below environment variable on your CLI -

DOCKER_BUILDKIT=1

With this new backend Docker predefines a set of ARG variables with the information pertaining to the platform of the node performing the build (build platform) and the resulting image (target platform)

The following ARG variables are set automatically:

These arguments are defined in the global scope so are not automatically available inside build stages or for your RUN commands. To expose one of these arguments inside the build stage redefine it without value as shown in below example:

Consider the below Dockerfile where we have all the above predefined ARG’s redefined

I carried out this build on a RaspberryPi to show all the values of above ARG’s

Read more about this here from Docker Documentation.

CE-EE node activate

November 8,2018 was a great day for Docker community as we saw new stable release for community edition Docker 18.09 CE afer moving to the new release and support cadence.This will be supported for 7 months with Docker 19.03 CE being the next release in line.

Along with this being an Enterprise customer we are excited to have Docker Enterprise 2.1 which adds support for Windows Server 1709, 1803 and Windows Server 2019. Read more about this in the blog from Docker team.

The one feature I was keen to test on was upgrading CE engine to EE engine using the Docker cli. Here are the steps I followed to upgrade a CentOS machine running 17.12.0-ce engine to 18.09 (EE engine).

$ docker version
Client:
 Version:	17.12.0-ce
 API version:	1.35
 Go version:	go1.9.2
 Git commit:	c97c6d6
 Built:	Wed Dec 27 20:10:14 2017
 OS/Arch:	linux/amd64

Server:
 Engine:
  Version:	17.12.0-ce
  API version:	1.35 (minimum version 1.12)
  Go version:	go1.9.2
  Git commit:	c97c6d6
  Built:	Wed Dec 27 20:12:46 2017
  OS/Arch:	linux/amd64
  Experimental:	false

Lets add the stable repository so that we have the latest pacakges to install the new release of Docker CE.

$ sudo yum-config-manager \

     --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Failed to set locale, defaulting to C
Loaded plugins: fastestmirror
adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo

To install specific version of Docker instead of always using the latest, firt list the available versions.

$ sudo yum list docker-ce –showduplicates

Failed to set locale, defaulting to C
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
docker-ce-stable                                      | 3.5 kB  00:00:00     
(1/2): docker-ce-stable/x86_64/updateinfo             |   55 B  00:00:00     
(2/2): docker-ce-stable/x86_64/primary_db                19 kB  00:00:00     
Installed Packages
docker-ce.x86_64        17.12.0.ce-1.el7.centos       @docker-ce-stable
Available Packages
docker-ce.x86_64        17.03.0.ce-1.el7.centos       docker-ce-stable 
docker-ce.x86_64        17.03.1.ce-1.el7.centos       docker-ce-stable 
docker-ce.x86_64        17.03.2.ce-1.el7.centos       docker-ce-stable 
docker-ce.x86_64        17.03.3.ce-1.el7              docker-ce-stable 
docker-ce.x86_64        17.06.0.ce-1.el7.centos       docker-ce-stable 
docker-ce.x86_64        17.06.1.ce-1.el7.centos       docker-ce-stable 
docker-ce.x86_64        17.06.2.ce-1.el7.centos       docker-ce-stable 
docker-ce.x86_64        17.09.0.ce-1.el7.centos       docker-ce-stable 
docker-ce.x86_64        17.09.1.ce-1.el7.centos       docker-ce-stable 
docker-ce.x86_64        17.12.0.ce-1.el7.centos       docker-ce-stable 
docker-ce.x86_64        17.12.1.ce-1.el7.centos       docker-ce-stable 
docker-ce.x86_64        18.03.0.ce-1.el7.centos       docker-ce-stable 
docker-ce.x86_64        18.03.1.ce-1.el7.centos       docker-ce-stable 
docker-ce.x86_64        18.06.0.ce-3.el7              docker-ce-stable 
docker-ce.x86_64        18.06.1.ce-3.el7              docker-ce-stable 
docker-ce.x86_64        3:18.09.0-3.el7               docker-ce-stable 

The standard way of installing a specific version is $ sudo yum install <FULLY-QUALIFIED-PACKAGE-NAME>

The version string is the package name plus the version up to the first hyphen. In the example above, the fully qualified package name is docker-ce-18.09.0

Let’s upgrade Docker to 18.09

$ sudo yum install docker-ce-18.09.0
Failed to set locale, defaulting to C
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 0:17.12.0.ce-1.el7.centos will be updated
---> Package docker-ce.x86_64 3:18.09.0-3.el7 will be an update
--> Processing Dependency: containerd.io for package: 3:docker-ce-18.09.0-3.el7.x86_64
--> Processing Dependency: docker-ce-cli for package: 3:docker-ce-18.09.0-3.el7.x86_64
--> Running transaction check
---> Package containerd.io.x86_64 0:1.2.0-3.el7 will be installed
---> Package docker-ce-cli.x86_64 1:18.09.0-3.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================================================================================================
 Package                                    Arch                                Version                                         Repository                                     Size
====================================================================================================================================================================================
Updating:
 docker-ce                                  x86_64                              3:18.09.0-3.el7                                 docker-ce-stable                               19 M
Installing for dependencies:
 containerd.io                              x86_64                              1.2.0-3.el7                                     docker-ce-stable                               22 M
 docker-ce-cli                              x86_64                              1:18.09.0-3.el7                                 docker-ce-stable                               14 M

Transaction Summary
====================================================================================================================================================================================
Install             ( 2 Dependent packages)
Upgrade  1 Package

Total download size: 55 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/3): containerd.io-1.2.0-3.el7.x86_64.rpm                                                                                                                  |  22 MB  00:00:07     
(2/3): docker-ce-18.09.0-3.el7.x86_64.rpm                                                                                                                    |  19 MB  00:00:09     
(3/3): docker-ce-cli-18.09.0-3.el7.x86_64.rpm                                                                                                                |  14 MB  00:00:03     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                               5.1 MB/s |  55 MB  00:00:10     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
  Installing : containerd.io-1.2.0-3.el7.x86_64                                                                                                                                 1/4 
  Installing : 1:docker-ce-cli-18.09.0-3.el7.x86_64                                                                                                                             2/4 
  Updating   : 3:docker-ce-18.09.0-3.el7.x86_64                                                                                                                                 3/4 
  Cleanup    : docker-ce-17.12.0.ce-1.el7.centos.x86_64                                                                                                                         4/4 
  Verifying  : 1:docker-ce-cli-18.09.0-3.el7.x86_64                                                                                                                             1/4 
  Verifying  : 3:docker-ce-18.09.0-3.el7.x86_64                                                                                                                                 2/4 
  Verifying  : containerd.io-1.2.0-3.el7.x86_64                                                                                                                                 3/4 
  Verifying  : docker-ce-17.12.0.ce-1.el7.centos.x86_64                                                                                                                         4/4 

Dependency Installed:
  containerd.io.x86_64 0:1.2.0-3.el7                                                      docker-ce-cli.x86_64 1:18.09.0-3.el7                                                     

Updated:
  docker-ce.x86_64 3:18.09.0-3.el7                                                                                                                                                  

Complete!

There are certain requirements to activate the EE engine from a CE:

1. The Docker Community Edition (CE) version must be 18.09 or higher.
2. All of the Docker packages must be installed: docker-cli, docker-server, and containerd.
3. A valid Docker EE license (obtain it from the Docker Hub under your organization profile)

At this point this feature has some limitations:

1. Only supported on x86 Linux nodes.
2. Windows nodes are not currently supported.
3. Node-level Engine activation between CE and EE is only supported in the same version of Docker Enterprise Engine for Docker.
4. Prior version of Docker CE do not support this feature

Considering the above requirements and limitation lets check the version of Docker to ensure that we can do a CE-EE node activation

$ docker version
Client:
 Version:           18.09.0
 API version:       1.39
 Go version:        go1.10.4
 Git commit:        4d60db4
 Built:             Wed Nov  7 00:48:22 2018
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.0
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.4
  Git commit:       4d60db4
  Built:            Wed Nov  7 00:19:08 2018
  OS/Arch:          linux/amd64
  Experimental:     false
 

NOTE: When running the command docker login, the shell stores the credentials in the current user’s home directory. RHEL and Ubuntu-based Linux distributions have different behavior for sudo. RHEL sets $HOME to point to /root while Ubuntu leaves $HOME pointing to the user’s home directory who ran sudo and this can cause permission and access problems when switching between sudo and non-sudo’d commands.

This is something I missed during my steps and thanks to Docker team for helping me out in resolving this issue.

Here is the new Management command available in this release:

$ sudo docker engine --help
Usage:	docker engine COMMAND

Manage the docker engine

Commands:
  activate    Activate Enterprise Edition
  check       Check for available engine updates
  update      Update a local engine

Make sure you have downloaded your license file from Docker Store to activate the license using the command below:

$ sudo docker engine activate --license docker_subscription.lic 
[sudo] password for sujaypillai: 
License: Quantity: 20 Nodes	Expiration date: 2019-07-30	License is currently active
18.09.0: resolving 
unable to pull image docker.io/store/docker/engine-enterprise-dm:18.09.0: failed to resolve reference "docker.io/store/docker/engine-enterprise-dm:18.09.0": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

The reason this failed is because you need to login to your dockerhub using the same userid which you used to download the license file.

$ sudo docker login docker.io -u sujaypillai
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

Lets activate the EE license

$ sudo docker engine activate --license docker_subscription.lic 
License: Quantity: 10 Nodes	Expiration date: 2019-05-25	License is currently active
18.09.0: resolved 
8426d869cd15: done [==================================================>]     433B/433B
04943fdcc3c7: done 
130b7807ec4a: done 
3470a7591bbb: done [==================================================>]  4.594kB/4.594kB
15bd6c72f8d4: done 
22797e024efe: done [==================================================>]    850kB/850kB
Successfully activated engine.
Restart docker with 'systemctl restart docker' to complete the activation.

Once you restart the system and execute the below command you may see the Server as “Docker Engine - Enterprise” while the Client is still on the Community

$ docker version
Client:
 Version:           18.09.0
 API version:       1.39
 Go version:        go1.10.4
 Git commit:        4d60db4
 Built:             Wed Nov  7 00:48:22 2018
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Enterprise
 Engine:
  Version:          18.09.0
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.4
  Git commit:       33a45cd
  Built:            Wed Nov  7 00:19:46 2018
  OS/Arch:          linux/amd64
  Experimental:     false

Switch between Linux and Windows Container on D4W

If you are using Docker for Windows there might be a need to switch between Linux OR Windows container at times.

The first way to achieve this is to right click on the whale icon from the system tray and choose either the option “Switch to Linux Containers…” OR “Switch to Windows Containers…”

The other way to achieve this from the CMD/Powershell is to execute the below command:

        & 'C:\Program Files\Docker\Docker\DockerCli.exe' -SwitchDaemon

There is a running process “com.docker.proxy.exe” which listens on the default named pipe : \.\pipe\docker_engine [for linux containers].

If you switch from Linux to Windows container the Windows Docker engine “dockerd.exe” will be started for you which listens on \.\pipe\docker_engine_windows & the above proxy process redirects to this.

Switch ORCHESTRATOR on D4W

If you are using latest Docker for Windows 18.02 CE Edge or higher it comes by default Docker Orchestrator as “Kubernetes”. For me at the time of writing this post I don’t see an option to switch it to Docker Swarm from the Settings menu.

If you ever need to change the Orchestrator to Swarm/Kubernetes you may set the environment variable DOCKER_ORCHESTRATOR to appropriate value as shown below

        set DOCKER_ORCHESTRATOR=swarm

Downgrading AMP version in Alfresco

If you ever tried to downgrade an AMP version in Alfresco you should have seen an error something similar:

There are two ways to solve this issue:

1. Configuring a rule on a folder to execute script action.

2. Executing a SQL update statement directly on the DB.

Configuring a rule on a folder to execute script action.

• Navigate to Admin Tools > Node Browser
• Change the “Select Store” drop down on the top right section to the option ‘system://system’
• Change the search option from “fts-alfresco” to “storeroot” and press Search
• This would give the workspace reference [system://system/c7cad4eb-98a2-4904-be3c-f6c98f83ad09]
• Navigate to /sys:system-registry/module:modules
• From the “Child Nodes” section find the module for which you need to update the versión and get the child reference in this case – system://system/31278e85-7fe5-49b2-a1b7-179707163da5
• Copy the following script in to a text file and save it as changeAMPVersion.js and save it under Repository > Data Dictionary > Script

var node = search.findNode("<nodeRef >");
node.properties["module:currentVersion"]="1.0.0";
node.save();

• Create a folder and apply a rule that executes the said script from the .js file uploaded in above step. Once executed the “module:currentVersion” of the module will reflect the required value.

If you have the Javascript console installed then you may execute the above script directly on JSConsole

Executing a SQL update statement directly on the DB.

• You may execute the below query directly on Alfresco DB to change the currentVersion value of AMP module

UPDATE alf_node_properties
LEFT JOIN (alf_qname, alf_namespace) ON (alf_node_properties.qname_id = alf_qname.id AND alf_qname.ns_id = alf_namespace.id)
SET alf_node_properties.string_value = "1.0.0"
WHERE alf_namespace.uri="http://www.alfresco.org/system/modules/1.0" AND alf_qname.local_name = "currentVersion" AND string_value = "1.0.1";

Aikau Audit Report

EisenVault takes away the hassle of managing documents by storing them in both electronic and physical forms, enabling smart-search and deploying quick retrieval measures. Our innovative cloud-based document management system can be used in multiple industry sectors, and caters to specific requirements of various job roles. Our vision - to be your preferred partner for electronic and physical document management.

This is EisenVault’s first Aikau plugin and we would like to share our experience building it. While during the days when people used Alfresco Explorer there was an option to view the audit trail for a document using the show_audit.ftl by selecting “Preview in Template”. Below is a screenshot of how it looked -

But there is no such option in Alfresco Share to view those audit data, so I thought of building it. Auditing is by default enabled in Alfresco but the data generation for the same is disabled and to enable this you would have to put in the below property in alfresco-global.properties.

        audit.alfresco-access.enabled=true

You can check the status for auditing by issuing the command:

        curl -u admin:password "http://localhost:8080/alfresco/service/api/audit/control"

On issuing the above command you would get a JSON response something similar -

    {
       "enabled" : true,
       "applications": 
       [
          {
             "name": "Alfresco Tagging Service",
             "path" : "/tagging",
             "enabled" : true
          }
             ,
          {
             "name": "alfresco-access",
             "path" : "/alfresco-access",
             "enabled" : true
          }
             
       ]
}

You can also query to retrieve the audit events:

        curl -u admin:password "http://localhost:8080/alfresco/service/api/audit/query/alfresco-access?limit=2"

{
   "count":2,
   "entries": 
   [
      {
         "id":1,
         "application":"alfresco-access",
         "user":"admin",
         "time":"2015-09-16T13:48:08.796+08:00",
         "values": null
      },
      {
         "id":2,
         "application":"alfresco-access",
         "user":"admin",
         "time":"2015-09-16T13:48:09.560+08:00",
         "values": null
      }
   ]
}

If you looking for more information about Alfresco Audting you should definitely look here

Aikau Audit Report page

• The new Aikau Audit report page lets you view the audit trail for each document in the repository as shown below:

  

• You can access this page either from the document library action OR from the Document Actions section from the document details page:

How does it work?

• The AlfList widget makes a call to webscript NodeAuditTrailWebScript with help of the attribute loadDataPublishTopic via the CrudService (using the “ALF_CRUD_GET_ALL” topic) which take the nodeRef as a parameter and this in turn returns the data in JSON format.
• Now to render the list what AlfList loads we need to create a view which is done with the help of AlfListView.
• As the list contains five columns we would be adding same number of HeaderCell for the attribute widgetsForHeader.
• Each row in the data will be represented by Row and each Cell will hold Property to render the value for respective Cell.
• As the last column “Audit Entry Values” holds multiple values the Property widget need to be overridden and you may find this in Property.js

** You may find the complete source code for this addon on Github

Alfresco Multi Folder Template Selection

If you look into the Email Form while setting an “Send Email” Action on a folder you would be wondering how we can add new email templates into the list of drop down “Use Template” as shown below -

The only option without any customization is to add your templates into the folder : Repository > Data Dictionary > Email Templates > Notify Email Templates

How does this work?

Look for the bean ‘ac-email-templates’ in alfresco/WEB-INF/classes/alfresco/action-services-context.xml which has the definition as below -

The property searchPath is what restricts it from fetching templates from a single folder.

What we need to change?

Let’s bring in our changes so that we can have multiple folders to select the templates from. For this we will be overriding the bean as below -

• The first change is converting the property searchPath from a String literal to a List.
• Then update the getAllowableValuesImpl() method so that it parses each folder path to search for what files to be included in the dropdown.
• I have also added an extra method sortByName so that the values in the dropdown are displayed in sorted order by their name.

Checkout the code for FolderContentsParameterConstraint for a better understanding of how it works.

I made this whole project as 2 step -

1. First is bootstrapping a template under a specified path. v1.0
2. Make the templates available for selection under Email Action. v2.0

The source code for this project lives here

Alfresco Tech Talk Live 85 - Global Virtual Hack-a-thon Project Demos

After the Global Virtual Hack-a-thon 2015 which was held on April 17, 2015 I was lucky enough to be on the panel list to demo the project what I worked on. Thanks Richard Esplin & Ole Hejlskov!!!

The panel included:

The Tech Talk Live this month was dedicated to review different projects which were carried out during Hackathon.

Below is the recording for the whole session:

During the hackathon I worked on converting Jeff’s Five Star Ratings Add-On to run on Alfresco Community 5.0.c using Alfresco SDK 2.0.

I had completed the Repo side work but still left with Share UI part as I am still learning AIKAU. If somebody in the community is willing to fork my project you may find it here .

Generating Checksums for Alfresco Content

Inspired from Chris Paul’s blog and work

I was reading about Alfresco Global Virtual Hack-a-thon scheduled to be on 17th April 2015 and going through the list of Projects came across - Document Checksuming and display in Share. My bookmark’s collection had Chris’s blog and thought of bootstrapping myself for hackathon by rewriting the whole work done by Chris using Alfresco SDK 2.0 on 5.0.c.

Below are the brief details about my approach:

• Create an empty directory which will hold both the repository tier & share amp archetypes.

• Navigate into the above created folder and issue the below command:

    mvn archetype:generate -Dfilter=org.alfresco.maven.archetype:

• You will be prompted to choose an archetype

• Enter 2 to create an AMP project for repository tier.
• For org.alfresco.maven.archetype:alfresco-amp-archetype version: choose value as 5 for selecting an archetype version of 2.0.0.
• Enter value for property ‘groupId’ as org.ootb
• Enter value for property ‘artifactId’ as chksum-repo
• The next step would confirm with all the properties set by you and if you are satisfied with it enter Y to create the project.
• Repeat the above procedure for creating a share amp project.

• Once you have both the projects created import them into eclipse.

• On the chksum-repo (repository amp) side I haven’t made any changes except that the below class where the name has been changed from HashableModel to ChecksumModel and the namespace url and prefix referring to OrderOfTheBee

package org.ootb.model;
import org.alfresco.service.namespace.QName;

public interface ChecksumModel {
    static final String CHKSUM_MODEL_URI    = "http://orderofthebee.org/model/checksum/1.0";
    static final String CHKSUM_MODEL_PREFIX = "ootb";
    
    static final QName ASPECT_HASHABLE = QName.createQName(CHKSUM_MODEL_URI, "hashable");
    static final QName PROP_HASHTYPE   = QName.createQName(CHKSUM_MODEL_URI, "hashType");
    static final QName PROP_HASHVALUE  = QName.createQName(CHKSUM_MODEL_URI, "hashValue");
}

• On the Share side to make the aspect visible to the user add the below part in share-config-custom.xml

<config evaluator="string-compare" condition="DocumentLibrary" replace="true">
 <aspects>
     <!-- Aspects that a user can see -->
     <visible>
        <aspect name="ootb:hashable" />
     </visible>
 </aspects>
</config>

This should make the Hashable aspect visible in the list of aspects available to add when the user clicks on “Manage Aspects” from document-actions.

• By default in the above screen it would display the aspect name as aspect.ootb_hashable and to add a meaningful value to it add the following bean:

<bean id="ootb.custom.resources" class="org.springframework.extensions.surf.util.ResourceBundleBootstrapComponent">
  <property name="resourceBundles">
     <list>
        <value>alfresco.module.${project.artifactId}.messages.chksum</value>
     </list>
  </property>
</bean>

• In chksum.properties file add the below line:

    aspect.ootb_hashable=Hashable

• Once you add the Hashable aspect to a document the policy handler would calculate the checksum (ootb:hashValue) by default using (ootb:hashType) as MD5 as shown below:

• To display the checksum value on properties page you need to add the following in share-config-custom.xml:

<!-- Display the Hashable properties -->
<config evaluator="node-type" condition="cm:content">
    <forms>
        <form>
            <field-visibility>
                <show id="ootb:hashType" />
                <show id="ootb:hashValue" />
            </field-visibility>
            <appearance>
                <field id="ootb:hashValue" read-only="true" />
            </appearance>
        </form>
    </forms>
</config>

• Status Indicator:

• Defined within the DocumentLibrary config section, status indicators are small icons typically used to indicate the presence of an aspect.
• Indicator images are by default reference by id: /res/components/documentlibrary/indicators/{id}-16.png unless the name is overridden by the “icon” attribute.

• As shown below any content which has an aspect oot:hashable applied to it (has a checksum value calculated) then it shows the icon “H”

• This is achieved by adding the following code snippet in share-config-custom.xml:

<indicators>
<indicator id="hashable" index="10" icon="hashable-16.png">
    <evaluator>evaluator.doclib.indicator.hashable</evaluator>
 </indicator>
</indicators>

• The evaluator evaluator.doclib.indicator.hashable helps to identify whether a node has an aspect oot:hashable and is defined in the bean

<bean id="evaluator.doclib.indicator.hashable" class="org.ootb.web.evaluator.doclib.indicator.ChecksumEvaluator">
</bean>

The complete source code for this example is available on GitHub.