Alteryx has a huge built in set of tools (about 170 tools in v10.1 I think), but when something is missing or you just have a specific way you want to do something, it has various options on how you can extend it.

The simplest way (and one of the most flexible ways) is to create a macro. These allow you to create a reusable workflow performing a specific task or calculation. The various types (Standard, Batch, Iteration and Location Optimisation) provide a huge amount of power on top the basic workflow. The designer makes it easy to take an existing workflow, convert it to take some inputs (via the Macro Input tool), and then produce some outputs (via the Macro Output). There is also a simple interface toolkit allowing them to have some configuration. The idea of this series of blog posts is to look at some other methods provided in the SDK rather than macros, but it is always worth bearing them in mind before diving into an SDK based solution.

The series will start with a overview of the Alteryx platform and the SDK, and then walk through how to create a XML Formula Add On extending the set of functions in the formula tool. After this I will talk about how to create some simple tests and an installation script for it. Then we will take a look at creating C++ based functions, before diving into creating custom tools in C#. I am planning to also look over the new HTML / JavaScript custom tools API (which were added in version 10.1).

All of this will be based on what I have learned from experimenting with the SDK. I’ll describe how I think it all plugs together. Ned Harding did a talk at Inspire on the internal structure of Alteryx – I am looking forward to hearing his Inspire Europe’s version and correcting all the mistakes in my understanding.

In general, I will use examples gathered from the questions asked in community or things I think are useful additions. All of the code will be available on GitHub – either as part of my Formula Add Ins or Custom Tools Add Ins. Hopefully by the end of the series, you will have learnt some very powerful ways to make your analysis easier and learnt some more about how Alteryx works.

Getting Started

The SDK is included in all Alteryx designer installs. It is in the APIs directory within the Alteryx install folder.

The AlteryxSDK.zip contains the C-style SDK for creating C/C++ custom tools and formula functions, but even more importantly it also contains the AlteryxSDK.chm documentation file. While you don’t need access to the contents in here to create either formula add-ins or .Net based custom tools, the information in the documentation file has been exceedingly helpful as I have built functions and tools. You will need to contact your sales team to get the password to extract the zip file.

The SampleCode folder contains the custom tools .Net API documentation and examples (as well as various automation examples using .Net). While this documentation was enough to get started – the additional detail provided inside AlteryxSDK.chm has been critical as I moved forward.

A Quick Overview of Alteryx Platform

A big proviso here – all of this is based on my experimenting with SDK and APIs and hence liable to be hugely inaccurate and missing vast sections as I only really know what I need to build my tool set. If you want the true structure and details then you will need to get to Ned Harding’s talk at Inspire Europe 2016.

Alteryx Designer itself consists of two main parts – the Designer and the Engine. The Designer is the front end the user interacts with to create a workflow that the Engine will run. A workflow is defined by an XML based configuration file which tells the engine what tools to create, how to configure them, and how they connect together. The diagram below shows some of the components of the platform:

The Designer

The designer has various core service and features. It is responsible for hosting the engine when running a workflow – collecting the results into Browse Anywhere data stores and log messages into the log buffer. Tools within the designer break into about three different types. A few areas are core functionality of the designer (the Interface tools for example) and are built into it (I believe these are only for things which a designer specific). These cannot be extended using the SDK.

Quite a lot of the tools are actually pre-built macros designed by Alteryx themselves. These act exactly the same as user created macros. You can learn a lot about what is possible and how to build macros by taking a look at these. If you go to Options => User Settings => Edit User Settings, under the Canvas tag there is an option to display an indicator to show something is a macro. You can then right click and view the inside of the macro:

Finally the vast majority of the tools come from plugins. Alteryx really seems to practice ‘dogfooding’ – using the SDK to build their own tools. If you go to the bin\Plugins directory within the Alteryx install path you will find a lot of different DLLs. These contain the core tool set of Alteryx.

The designer hosts a collection of DLLs, containing one or more plugins. Each plugin has a configuration designer and an entry point for engine to call. The entry point does not need to point to the same DLL or even be based in the same technology as the configuration GUI. This means you can use the platform which suits you best to design the GUI (either .Net or HTML) and use a different one for the computation (e.g. C++). Historically, Alteryx have used C# for their GUI controls and C++ for the engine side, though I expect to see more HTML based GUIs in the upcoming versions.

One additional part of the designer which is useful to be aware of when working with the SDK are is Default setting file. This tells the designer how to arrange the ribbon and layout the tools. While not technically part of the SDK should you want to put a space in your tool’s name the only way is in this configuration file. It is in the bin/RunTimeData folder and called DefaultSettings.xml.

The Engine

The engine is the magic of Alteryx. It is an exceptionally fast platform for working with data. It is hugely expandable – for example the In DB library is a Engine Plugin adding a completely different way of interacting with a database than was there before, and it can cope with massive volumes of data (even on relatively modest hardware):

I am not exactly sure where in the process the Xml in the workflows and macros are decoded (I guess in the Engine itself), but the engine converts it into a calls onto functions in the Engine Plugins. The Entry Points specify the name of DLL and the name of the method to call. The Xml specifies how to the data flows through the workflow.

The formula tool is itself just part of one of the base plugins. It however adds some additional features to the SDK. The XML formula add ins allow you to add additional functions either as simple XML macros or as references to C-style functions.

Other Parts

On top of these two parts, there is also a command line interface to execute workflows (AlteryxEngineCmd.exe) and the server product. I have no experience of the server product and only very limited experience of the AlteryxEngineCmd (I had brief access to it at my previous job). All of the techniques from the SDK work in both these environments I believe although only those relating to the engine are relevant, but I have never used them in anger.

The SDK

There are three versions of the SDK: C-Style, .Net API and the new HTML/JS API. One fantastic feature is you can mix and match as mentioned above – e.g. use HTML to define the GUI and C++ for the engine. I am not sure if 100% free choice – when I get to writing about the HTML API will try calling a C# engine plugin!

C-Style SDK

The C-Style SDK allows you to create either functions for the Formula tool or new engine plugins. In this blog series, I will take a look at using this SDK to create new functions using just XML and also using C/C++ based functions. I am far from an expert on C++ so have not tried using it to create Engine Plugins with it, but this is the platform Alteryx themselves use. The plugins for the designer are .Net based (or HTML based) so the C/C++ SDK wont help with these.

The SDK also provides the ability to run the engine from your own C code. I do not have the appropriate license for this or any experience with using it.

This is the fastest and most native of the engine APIs.

.Net API

The .Net API allows you to create custom tools both the GUI side and the engine side. There are some limitations on what is possible with this API (no access to Browse Anywhere for instance, and I reckon no way to do something like In DB tools via it!) but it is very flexible and allows you to create tools that act and feel like first party ones.

The formula functionality of the C-Style SDK is only available in C.

As with the C-Style SDK, there is an automation style add in allowing you to run the engine and workflows from your own .Net code. Again you need a special license to use this functionality so will be beyond the scope of this series.

HTML/JS

The HTML API is the newest. It has the ability to create custom tools both the GUI and the engine side. This is the furthest away from the native engine code so I expect performance to be slower. That said if you want to read data from web pages it is going to be by far the easiest.

I am still playing with this API but hope to include it as part of this series. There is also some interesting work going on about how to package and release these (I hope it will be extended to cover the older SDKs too), which might make them more appealing within the community context.

Next Post…

First up, will be creating pure XML based, formula functions.