Author: Edward Caulfield
Release Date: May 4, 2017
- 1 What is IoT?
- 2 The Open World
- 3 ThingWorx Foundation
- 4 ThingWorx Studio
- 5 Extended Tracking
- 6 Installation
- 7 Creating a Project
- 8 The Project Elements
- 9 Configuration
- 10 Views
- 11 Canvas
- 12 Working With Models
- 13 Events and Services
- 14 Managing the 2D Overlay
- 15 Manipulating Objects
- 16 Communicating with Devices
- 17 The Composer
- 18 Behind it All
- 19 Summary
- 20 What’s to like?
- 21 What’s to not like?
Many years ago there was an entertaining television commercial in the USA where a company, I think it was Wendy’s, explained how much care and effort they put into making a perfect meal for their customers. Each and every focal point that showed the extra care and effort they took was followed by some lame guy saying “Boy, you sure make a lot of work for yourselves!!” At the time I didn’t realise how many truisms this commercial contained, but the implied message of “Good things take extra effort” didn’t escape me and with time I learned to appreciate how much work really goes into making extraordinary things.
PTC also seems to have taken this philosophy to heart. While it is very easy to find large software products that are complex, difficult to use, and bug-ridden, it is a rare treat to find a powerful tool that does a great deal of heavy lifting for you in an elegant and reliable manner. For my dime, PTC has done just that with their ThingWorx ensemble. They have made a lot of work for themselves, so that you and I don’t have to.
Although there are numerous vendors offering IoT solutions, ThingWorx is the only one I have found that focuses on developing IoT interfaces with a minimum of coding skills required and has an Augmented Reality design interface. Considering the time and effort that has gone into the development of ThingWorx Studio, I expect it will be a quite some time before any IoT competitors match or exceed the ThingWorx platform capabilities.
ThingWorx is more of a concept than an application suite. It works very diligently to create an ecosystem for IoT devices to be created, deployed and maintained with a minimum of effort and a maximum of consistency and reliability. Whereas in the “old world” vendors would create proprietary solutions, forcing you to live within the boundaries of their technology, ThingWorx opens the world of IoT to allow companies to pick and choose Best-of-Breed solutions for the specific needs that they have. No longer should you be locked into proprietary technology, tying your fate and fortune to that of your supplier.
What is IoT?
The world has started to deliver an avalanche of devices that fall into the category of “Internet of Things”, IoT for short. To be fair, IoT is just a rebranding of a technology concept that has been around for literally decades. In its most basic form, IoT is the idea that machines need to be inter-networked so that they can collect and exchange data amongst themselves to do work without constant human intervention. People can be engaged to address escalations and solve problems that arise, as well as monitor overall operational behaviour, but shouldn’t need to be involved in normal day to day operations.
What makes IoT difficult is that there is such a broad array of technology components, that no single vendor can provide everything that you might need to build your end product. Thus, your ability to build great products is either limited to a given vendor’s technology offering, or you have to do the ugly work of creating an interface that will allow all the different technologies to work together as an orchestra. Either one of these can be a nightmare.
With ThingWorx, you can define the interface to your Thing and allow this to be immediately consumed and used to create great products. Anyone running on ThingWorx then gets an almost effortless integration of components when they are delivered with ThingWorx support.
The Open World
It has not been lost on most technology leaders that one of the most important reasons for the explosion of the Internet and surrounding technologies is the fact that there are well defined standards and interfaces to enable things to work. It doesn’t matter which browser I use, the underlying cascade of protocols CSS, HTML, HTTP, UDP, IP etc., ensures that everybody knows how to communicate with the layers above and below in a consistent and reliable manner.
The first thought that crosses my mind when it comes to openness in hardware is that it gives me flexibility to exchange widgets in my design without having to rewrite any code. The reality is, however, that seldom are two widgets identical in all aspects. If the physical packaging isn’t different, then the operating characteristics probably are. So, while an open system can make it easier to be less dependant upon any given supplier, the real benefit is in the ease with which components from various suppliers can be integrated into a new design. You no longer have to bother about the design interfaces for each component and can focus your energies on making an outstanding product.
Historically, regardless of what their marketing team says, most vendors haven’t been too thrilled about the idea of an industry standard design interface. A lot of effort goes into creating an environment that will lock a customer into a product portfolio. The higher the wall and the wider the moat, the more difficult it is for customers to escape your grasp. The flip side is, of course, that it then can be more difficult for new customers to get into your ecosystem.
Who benefits from a standardised design interface? Market entrants and customers who build using the standard. Market entrants benefit because they can create a device that can easily be implemented using a common design interface and customers benefit by reducing their effort to integrate parts from numerous suppliers into a design. While large incumbents will be all smiles when they join the party, I expect that they are grumbling and dragging their feet all the way there.
PTC has created the ThingWorx Foundation with the stated goal:
ThingWorx Foundation is a complete, end-to-end technology platform designed specifically for the Internet of Things. It empowers developers to connect, create, and deploy breakthrough, enterprise-ready IoT applications, solutions, and experiences.
ThingWorx Foundation connects to all of the ThingWorx components, providing a simplified, seamless approach for developers to create comprehensive IoT solutions.
A very important part of the ThingWorx ecosystem is their inherent Augmented Reality capabilities. A while back AR Intelligence reviewed a number of Easy AR CMS tools which allowed for the creation of AR experiences without writing a single line of code. ThingWorx was not included in this evaluation because it requires the installation of a local application and it goes a step beyond tools which allow you to overlay AR experiences on images and targets. ThingWorx will not just show a generic AR experience for a given marker. It is capable of tying that marker to a specific device and obtaining information from the device being viewed. It can also be programmed to actually interact with the device. While all of the platforms evaluated as part of the Easy AR CMS Product Review had SDKs which would allow full programming functionality, ThingWorx is unique in that it goes much further in its attempts to create a legitimate Drag-and-Drop configuration tool with an absolute minimum of programming required. They call this the “Democratisation of IoT” and I think they fulfil their promise very well.
Obviously, there is the implicit drawback that once you jump on board the ThingWorx bandwagon, you now tie your fate and destiny to ThingWorx instead of your component supplier.
Because a full evaluation of ThingWorx would take innumerable pages and this site is focused on Augmented Reality, this article will focus on ThingWorx Studio, which is the interface that ties the the IoT device to Augmented Reality experiences. To begin with, it is important to note that ThingWorx is marker oriented and does not have an object recognition engine, also known as marker-less recognition. What this means is that each device included in the IoT landscape will need to have its own unique marker. In large environments this can lead to literally hundreds, if not thousands, of markers and brings with it its own potential administration nightmare. It is possible to simplify things by having a serial number plate printed with a ThingMark, however the requirement of a unique identifier per device remains.
Because ThingWorx is designed to be able to be used in industrial environments, the markers need to be kept visible and undamaged. While the ThingWorx scanner will allow for some occlusion or distortion, everything has limits. If markers are implemented in physically hostile environments, care will have to be taken to ensure that they remain usable throughout their implementation lifetime. You will want to proactively identify and correct damaged markers before you get yourself into a situation that desperately needs the marker to be readable, and it is not. Keeping markers clean and readable clearly adds to instrument maintenance overhead, but I expect that the productivity gains realised with Augmented Reality more than make up for this.
Personally, I would like to see some flexibility on the ThingMark requirement. OCR is a longstanding and reliable technology, so why not just scan an existing Serial Number plate? These are almost always easily visible and usually made from very robust materials that stay readable for the life of the instrument. As well, RFID could be used to identify the device. While the administration overhead of one RFID tag per instrument remains, you no longer have to worry about making sure the tag is clean and visible, just readable.
Alternatively, for devices that are permanently installed in a single location, it could be possible to use a ThingTemplate to identify a device type, then use indoor geolocation functionality to identify your physical location down to the centimetre, and from this determine the specific instance of the device. For that matter, precise geolocation functionality could get rid of the need for a marker at all.
As ThingWorx is a work-in-progress with very regular releases, it will be interesting to see what additional functionality is added over time.
ThingWorx implements a functionality referred to as Extended Tracking, which utilises features of the environment to improve tracking performance and sustain tracking even when the target is no longer in view.
Extended Tracking allows the tracking a degree of persistence once a target has been detected. As the target goes out of view, ThingWorx uses other information from the environment to infer the target position by visually tracking the environment. ThingWorx builds a map around the target specifically for this purpose and assumes that both the environment and target are largely static.
Although ThingWorx Studio is a browser based application that is currently only fully supported under Chrome, it runs using a server hosted on your own computer. This requires that you download an application onto your computer and run it whenever you want to use ThingWorx Studio. This then opens up a browser with the ThingWorx Studio interface which is currently pre-populated with two demonstration Projects.
Having your project managed via a local computer has the advantage that you are not burdened if you have a slow or unreliable internet connection, however it does create a local footprint on your computer and takes up a few hundred MB of space. Considering the average size of a modern hard disk or SSD, the space taken by ThingWorx Studio is trivial and the tradeoff is a good one. As well, considering the low level at which the ThingWorx Studio Server is working, I doubt you will ever run into significant OS compatibility problems.
While each ThingWorx Studio Project can have any number of markers and corresponding IoT elements, the interface seems to be designed to work best when there is only one IoT device defined per Project.
Creating a Project
Creating a Project is in ThingWorx Studio is very easy. Simply click on the green circle at the top right hand side of your browser.
When you do this, you will get a dialog asking for some basic information.
The Project Name is required and can be just about anything that you want. The Server is actually a value provided automatically by ThingWorx Studio. You then select whether you want an AR or 2D experience and click on Create. The differences between the 2D and AR Projects are as follows:
- 2D Projects allow only 2D elements to be displayed in the experience and once the marker is scanned, the video capture is stopped and screen is filled with the 2D panel defined in the Project. The 2D panel is a single plane onto which objects are deposited and positioned.
- AR Projects have 2D and 3D panels. The 2D panel is a display layer that corresponds to your viewing screen and is transparent throughout the AR experience. The 3D panel allows the display of a 3D object that is tied to a marker. The object can play animation sequences, such as showing how a device is disassembled and reassembled. The 2D panel of the AR Project is broken into five regions – top, bottom, left, right and center.
For the purposes of this article we are only considering AR Projects.
The Project Elements
When you create a new AR Project, you are put into the Project Editor and you will see something like the following:
On the top left you have your Project Name. You will notice that nowhere on the screen do you have an Exit or Cancel button. Leaving the Project is accomplished by clicking on the “My Projects>” button to the left of the Project Name.
Your screen is broken down into four sections, the Center which contains your Canvas, Widgets and Details,
the Left which contains your Project Configuration, Views, Styles and Resources,
the Bottom with Widget bindings,
and the Right, which contains Data in the form of Application Parameters and External Data.
Visibility of the Left, Bottom and Right views can be toggled by clicking on the corresponding View on the top right of your browser window.
Starting with the Left Panel
your Configuration is broken down into Themes, which is a combination of Styles,
Experiences, which is where you tie your Project to a given marker
Info, which is simple administration information
and My ThingMarks, which is a collection of markers available to you.
You will be working a great deal with the Views panel. This is a listing of all the 2D and 3D elements found in your project and is where you connect parameters, events and actions to Widgets.
The AR Project will automatically create a top level View called “Home” and populate this with a 3D Container, which is then populated with a 2D Overlay. You can quickly see how the 2D Overlay is broken down into a Top Panel, a 2D Body and a Bottom Panel. The 2D Body is also broken down into a Left Panel, a Center Panel and a Right Panel. Whatever Widgets you place into each Panel will be shown under the View hierarchy. Every Widget that you create will need to be deposited in either directly into the 3D Container or somewhere in the 2D Overlay.
For the example below, I have put a Grid Layout Widget into the Bottom Panel, giving the Widget two rows and two and three columns for each row, respectively.
Notice that the column names are sequential to the order in which they are created. When the Widget was created, it had one row and one column. When I added a row, it added the second column. When I went back to row one and added a column, the third column was created. When I went back to the second row and created two more columns, the fourth and fifth columns were added.
To improve clarity, you can always change the names of any Widget or Object in the View.
By clicking on the green plus sign to the right to the Views label, you can create any number of new Views, which all have the same default contents described above.
The purpose of allowing for multiple views is to give the developer a means to create more complex Augmented Reality experiences. For example, if the user needs teardown / assembly instructions for a given device, and it has elements within it that can have their own teardown / assembly instructions, you can use separate Views to display the different sets of instructions.
The Canvas is where your visual work takes place.
The real power of ThingWorx lies in its ability to display 3D Models and play animations of these models. In the 3D Container you can deposit:
- 3D Gauge
- 3D Image
- 3D Label
- Model Item
- TML (ThingWorx Markup Language) Text
Although you can drop any item onto the Canvas in any order, you will need to always have a ThingMark present. The ThingMark is the tag that is placed on an instrument and used to associate the AR experience with the physical device.
Although the Gauge, Image and Label are called 3D elements, they aren’t really 3D objects. They are simple 2D objects which can be placed in the 3D space. These objects are most commonly used to display information relevant to the 3D Model that is displayed.
Working With Models
Although it is very easy to work with Models in ThingWorx, I am not finding it intuitive. Fortunately, ThingWorx has a very good set of basic tutorials to get you started.
A Model can be thought of as a container that holds a 3 dimensional model, which can contain any number of elements, and any number of Sequences. Once you drop a Model onto the Canvas, you will need to select the Resource that defines the model.
You can either upload the Resource directly from the Model Properties, or you can add the Resource to the Project’s Resources and select it in the Properties.
The Model can also have a Sequence, which is an animation that is applied to the object. One example of a Sequence would be a teardown of the Model displayed.
Events and Services
Each Model responds to a series of Events and provides a list of Services. Events and Services are how you communicate with the Model and its Sequence.
Services can be thought of as commands that the Model will respond to and are usually requested using buttons or toggles that you have placed on the 2D Overlay.
Events can be thought of as triggers that can be sent from the Model to other 2D or 3D Objects.
For example, let’s say that we have placed a Play Button on the 2D Overlay. We can start out by having the Play Button invisible, making it visible only after the Model Loaded Event is fired and sent to the Play Button. Upon being clicked, the Play Button would send a request to the Model’s Play Service that would initiate a Sequence. When the Sequence is started, it could send an Event to the Play Button to force it to become invisible and for a Stop button to become visible.
Managing the 2D Overlay
The key difference between the 2D and 3D displays that ThingWorx uses is that while the 3D display will always automatically rotate and adjust depending upon where you stand in relationship to the ThingMark, the 2D display is always specific to your display screen. That is, a 2D Widget that is placed in the Top Panel of the 2D Overlay will always been shown at the top of your screen. This makes it easy to configure Widgets that stay put on your screen, no matter what is happening to the 3D Object.
ThingWorx offers a long list of Widgets that can be added to the 2D Overlay:
- Bar Chart
- Card (which can hold Objects)
- Data Grid
- Grid Layout (which can hold Objects)
- Resource Image
- Text Area
- Text Input
- Time Series Chart
- Toggle Button
- Value Display
I will leave it to the reader to visit the ThingWorx documentation and review the functionality of each of the Widgets.
Each 3D Object can be manipulated by the user if you wish. This is accomplished by creating an Application Parameter that will hold the value of a given characteristic, and then tying that Application Parameter to a given control object. When you tie the Application Parameter to the Object, you will have to tell the system which characteristic it is tied to.
- Create an Application Parameter
- Tie the Application Parameter to a 3D Object by dragging it to the Object and selecting the Object’s characteristic that will be controlled
For example, if you had a 3D Model that you want the user to be able to rotate horizontally and vertically, you would create two Application Parameters to store the horizontal and vertical rotation value and tie these to individual sliders that would rotate the model from 0 to 360 decrees using whatever increment you desire.
I would personally prefer to be able to rotate Model using swipes, as a default function of the application.
Communicating with Devices
All of the above is fine and wonderful, but the whole point of creating ThingWorx Studio applications is to be able to communicate with networked devices. This is accomplished by using the External Data interface.
By clicking on the “+” to the right of “EXTERNAL DATA”, you are prompted for the Thing that you will be wanting to communicate with. A Thing can be thought of as a data structure that defines the parameters and functions associated with a device, and for object oriented programmers can be most readily understood as an instantiation of an object. For example, the object could be called Car and have hundreds of sub-components, such as environmental control, entertainment, navigation, safety, etc. If your Thing is a Car you would have to drill down into the structure to get to a Fuel Pump, for example. You could also have a Thing that is a Fuel Pump and access this device directly.
When you add external data, you will need to select the Thing you are adding and then specify if you are adding Properties, Services or Events.
A Property is simply a characteristic of the Thing, such as the Engine Temperature or Oil Level, as shown in the example above. Services are functions that can be called on the Thing that will either retrieve complex data structures or change the Things’s operational behaviour, such as Toggle Headlights in the example below.
Events can be thought of as alarms that you can subscribe to when something happens with the Thing. For example, you could be notified every time the status of the Cruise Control or Headlights change and reflect this information in your Augmented Reality experience.
So, where do Things come from? They are created in the ThingWorx Composer, which is a very powerful configuration tool. I think it is fair to say that you can do just about anything you would want with the Composer and considering how complex the Composer is, I ran into only a few aspects which I felt could be improved.
The Composer has its share of trivial problems, like:
- Being able to add objects that I had already deleted from the system,
- Not being able to delete objects because they are referred to, but not telling me directly what is referring to them
- Drag-and-drop functionality not always working, but not giving you error messages to tell you what you’re doing wrong
Explaining the Composer is beyond the scope of this article, so I suggest that if you want more information you simply sign up for a demonstration account on ThingWorx and spend some time working through their examples. ThingWorx has demonstration licenses for their Foundation Server, which gives you access to the Composer and can either be hosted by ThingWorx (30 day demo available) or downloaded to a local server (120 day demo available).
Behind it All
The whole point of ThingWorx is to enable you to communicate with IoT devices and there are numerous ways of accomplishing this task:
REST API – The ThingWorx Platform is fully exposed using web services so every property, service, subsystem, and function, is accessible using a RESTful API.
Edge Microserver (EMS) – The EMS is a pre-built application that you can install on Windows or Linux. It serves as a bridge between the remote device and the ThingWorx Platform, using the REST API AlwaysOn protocol. The EMS comes with the Lua Script Resource, which serves as an optional process manager, enabling you to create properties, services, and events in a scripting language called Lua script.
SDK’s – ThingWorx SDKs enable you to create applications that connect a device to the ThingWorx Platform. All of the SDKs communicate using the AlwaysOn protocol, which is light, efficient and firewall friendly. They enable you to execute services on the ThingWorx Platform and to create remote things that have properties, services, and events.
There are several different ThingWorx SDK Platforms:
- The C SDK, distributed in source code
- The Java SDK provides Java APIs.
- The Microsoft .NET SDK may only be used on Windows based machines.
- The iPhone Operating System (iOS) SDK integrates with Apple iOS based devices like the iPhone, iPad, and iPod Touch.
- The Android SDK may be used similarly to the iOS SDK.
ThingWorx provides a comprehensive, mature and stable platform to develop Augmented Reality enabled IoT functionality with a comparably short learning curve.
If you want to deploy IoT devices with a minimum of software development expertise, then ThingWorx has to be high on your list of development tools. If you want native Augmented Reality capabilities for your IoT development, it is the ONLY tool on your list – right now, at least.
A very valid question that I expect will be asked is “Is ThingWorx Studio a solution in search of a problem?” That is, can you do anything legitimately useful with this complex tool? I think the answer is clearly yes, but it won’t be immediately obvious to everybody. As with the invention of the semi-conductor, PCs, Faxes, the Internet and Email, there will be plenty of folks who look at the new tool and say “So what?!” It’s not until we start working with a tool that we begin to understand it’s value and that value is different for everyone.
What’s to like?
- Very robust and powerful tool that enables easy Augmented Reality experiences in IoT environments with very little programming
- User friendly interface
- Undo / Redo / Copy / Paste functionality (surprisingly rare in AR tools)
- Documentation has good examples and is written in an easily understandable manner
What’s to not like?
- No inherent team or revision control capabilities
- No object recognition
- ThingWorx Composer doesn’t match the user friendliness of ThingWorx Studio
- On-line documentation incomplete, sometimes dying half way through a topic, and not spell checked
- Poor debugging facilities
- No pricing information available