Page Comparison

Widget Connector
width 560 url https://www.youtube.com/watch?v=Cs37Mw-Q1PE&feature=youtu.be&allowfullscreen= height 315

There are a number of improvements in the March 2018 Release of Kitewheel. The highlights include: 

• Schema Navigation - making it easier to work with large schemas
• S3 File Adaptor - read and write files from AWS S3 buckets 
• Kitewheel Graph API Consumes XML - the graph API will now accept and parse XML 

• Kitewheel

  Hub Template Synchronizaton Control - for OEM partners 

  Hub Front-End Performance 

• Web Tracking Do-Not-Track Features
• Cross-Engine Visual Testing 
• Template Synchronization Control

There were a number of other bug fixes and minor improvements made during this release. We hope you like it. 

Schema Navigation 

One of the most common issues that our users have raised is the ability to easily find schema items once the schema grows to any level of complexity. We often encounter this within real projects where finding the correct schema elements, particularly if there are siblings with the same children, can lead to confusion. We have heard your pain and this is the first improvement in the user interface - we have many more planned. 

The Schema now has a Filter Schema search input box at the top directly below the Schema action buttons. Anything you type in there will be used for a substring search on any of the records or element names in your schema. Single or multiple words can be used and all of the words are searched separately . (for example, filtering by using 'email twitter' will return all values that container either email OR twitter OR both).

Schema elements are now shown with vertical indicators to help with deciding the child level: 

...

• Highlight all parents that match - whilst leaving their children expanded
• Highlight all children that match - whilst leaving parents expanded if they match, or collapsing them if they don't do not and just showing show the path 

Any schema elemented already selected (and thereby able to be used by a node or adaptor as a data source) remains selected

The Twitter status schema is sufficiently large and complex to demonstrate all of these actions: 

1. The word "tweet" has been typed into the schema search box
2. The top-level schema element "tweet" was already highlighted selected in the schema, and remains selected in addition to being highlighted
3. The child fields "retweet_count" and "retweeted" have been highlighted along with the parent field "retweeted" status 

...

To clear your search filter press the Clear Search Filter buttonbutton Image Added.  Image Removed

S3 File Adaptor 

Amazon Web Services Simple Storage System (S3) is an object based storage scheme that presents a file-system view of the objects that it stores. Folders are termed buckets and files are objects. To all intents and purposes it appears like a web hosted file system. The ability to read and write data from S3 buckets is useful for integrating with other systems that can only hadnle handle file-based integrations such as DMPs. 

...

The File Connector currently only supports Amazon S3 as the Type (we plan to extend this to Azure Blob Storage soon). The Connector requires the S3 Bucket name which is the top-level bucket name without any additional path, and the AWS Access Key and Secret Access Key.

...

The File Connector provides a Test Connection button consistent with the other connection types. 

Creating a File Adaptor

To create a file adaptor (for reading or writing), select the File option from among the adaptor types when creating a new item

Image Added

Writing a File to an S3 Bucket

In order to write a file to an S3 file a filename, including path, and an object that represents the contents of the filebucket you need to provide a File Name (including any path) and a File Contents Source: 

The File Name value can be provided by a literal, schema value or public variable. The file is created on each write and will overwrite any existing file without creating an error. It is therefore suggested that a unique identifier is used to create the filename. Often buckets contain many thousands of small objects rather than one large file - most third party data loading processes are happy to process a large number of small files. 

...

A JavaScript Node can be used to generate a file name that uses a timestamp and a unique identifier to ensure that files are ordered by creation date and also that there is no clash or accidental overwriting if multiple engines are processing and writing files simultaneously. The TransactionId from Schema Metadata can also be used to get a unique identifier. 

Reading a File From an S3 Bucket 

In order to read a file from an S3 bucket you need to provide a File Name and a File Contents Destination: 

Image RemovedImage Added

The File Name should include the path to file and filename. The Contents Destination is the location for the body of the file. 

...

There is currently no way to list all of the files in a bucket - but this is something that we plan to add. A sample use case may be to have a stream of file names that require to be processed and the file reader procesisng processing those files. 

Kitewheel Graph API Consumes XML

...

• XML elements become JSON arrays
• XML attributes becomes become "$" object objects with fields
• XML body text becomes "_" when there are attributes
• All element names are flattened to lowercase
• Attribute names are not flattened

<xml/>

{ 
  "xml" : "" 
}

<xml>
  <element/>
</xml>

{ 
  "xml":  {  
     "element": [ "" ]  
   } 
}

<xml>
  <element>a</element>
  <element>b</element>
  <element>c</element>
</xml>

{ 
  "xml": { 
    "element": [ "a", "b", "c" ] 
  }
}

<xml>
  <element attr1="a" attr2="b">first element</element>
  <element>second element</element>
  <element/>
</xml>

{
    "xml": {
        "element": [
            {
                "$": {
                    "attr2": "b",
                    "attr1": "a"
                },
                "_": "first element"
            },
            "second element",
            ""
        ]
    }
}

<xml>
  <element attr1="a" AttributeTwo="b">first element</element>
  <element>second element</element>
  <element/>
</xml>

{
    "xml": {
        "element": [
            {
                "$": {
                    "AttributeTwo": "b",
                    "attr1": "a"
                },
                "_": "first element"
            },
            "second element",
            ""
        ]
    }
}

<xml>
  <element attr1="a" attr2="b">first element</element2>
  <element>second element</element>
  <element/>
</xml>

{
    "message": "Unexpected close tag\nLine: 1\nColumn: 55\nChar: >",
    "error": {}
}

Kitewheel Hub Front-End Performance

...

One of the most important benefits that we expect is an overall performance improvement in the application itself. Benchmark tests show that React 16 is about 20% faster in the browser. Also the actual package itself is smaller. This should lead to faster load times and a more responsive experience for all. 

If for some reason the Kitewheel hub application gets into an unexpected state then you might see an image similar to the ones below.  For the first one, a reload of the page (app) will usually allow you to continue without encountering other problems.  For the second, navigate to a different area (without reloading) will usually allow you to continue.  

Image Added

Image Added

If you do see an error message like this then there is usually a meaningful error that we have printed in the browser console. If you know how to get to your browser console then please send us a screenshot of what you see to support@kitewheel.com as this will make it easier for us to diagnose the problem.

As part of this upgrade we have improved the color chooser to make it easier to use and to interact better with the local operating system. This is an example of the color chooser used to add some color when creating a tag: 

...

On Windows the operating system color choose will bring up the Windows color chooser: 

Web Tracking Do-Not-Track Features

The Xponent Graph API web tracking JavaScript has been modified to add a "doNotTrack" configuration section that allows the specification of HTML DOM elements and cookies that should not be tracked. 

• cookies - the cookies specified will not be returned in the web tracking events
• names - any HTML DOM elements that match the list of names will not be returned
• ids - any HTML DOM elements with the specified ids will not be returned
• classes - any HTML DOM elements that have this class will not be tracked 

The doNotTrack option is currently only available by modifying the JavaScript _kw.options object. There is no UI configuration available as part of this release. 

For example if you had a HTML body like this: 

  <body>
    <h1>Hello world!</h1>
    <div id="content">
      <code id="response">This is a div tag</code>
    </div>
    <div id="testers">
      <a href="index.html" class="doNotTrack">This is a link!</a><br/>
      <div id="form">
        <form action="index.html" method="post">
          <input type="text" name="myText"/>
          <input type="password" name="passField"/>
          <input type="submit" value="Submit me"/>
        </form>
      </div>
    </div>
  </body>

Template Sync Control 

...

The following "doNotTrack" object could be used within the _kw.options object: 

_kw.options = {
    "listenerId": "0123456789000000",
    "listenerHost": "https://api.kitewheel.com/api/v1/listener/",
    "doNotTrack": {"enabled": true, 
                        "names": ['passField', 'field1'],
                        "ids": ['content', 'passID'], 
                        "classes": ['doNotTrack'],
                        "cookies": ['privateCookie']
                        }
  } 

This would have the following effect: 

• The password field would not be tracked in a blur or a formSubmit event because of the names setting 
• The content div will not be tracked 
• The link will not be tracked as it has the "doNotTrack" class - this is the recommended approach
• The cookie "privateCookie" will not be returned as part of any packet 

Cross-Engine Visual Testing 

Kitewheel has supported remote engine stacks for some while. However it was not previously possible to perform visual testing with a remote stack. In this release we have enabled visual testing for remote stacks. 

A remote engine stack is any engine configuration that is not the default engine configuration for the Kitewheel Hub. This includes dedicated production stacks within the Kitewheel AWS environment but also in remote AWS and Azure environments that are managed by Kitewheel. Remote engine stacks are used for production workloads to provide dedicated throughput and security for connection to on-premise or other cloud environments. If you are interested in using a remote stack for these purposes please speak to your Kitewheel account manager. 

Template Synchronization Control

In this release we have improved the way that templates synchronise between different Kitewheel and dedicated partner OEM hubs. This makes the process more robust and gives a much finer level of control on how templates and managed graphs are distributed. 

Bug Fixes

• [KIT-1246] - Update Migration Strategy for Engine / Reporting DB to avoid potential database locks 
• [KIT-1317] - Custom engine group settings for environment disappear on a version restore
• [KIT-1330] - Page freeze (shows error boundary display) on organizations admin if navigating immediately after login

Story

• [KIT-9491376] - S3 Storage Adaptor: write, read (possibly) and no listenWebtrack: Tag attributes missing in data return object when event tracked is not a tag
• [KIT-10531307] - Upgrade to React 16Fix for large schema elements not passing correctly to JavaScript microservice
• [KIT-11661321] - Give the user manual control over where Graphs/Templates are synced toDuplicate deployments on Azure / Kubernetes stacks
• [KIT-11961354] - Kitewheel Hub needs the concept of an "Organization Operator" userSchema filtering fails with arrays in the Schema
• [KIT-1230] - Add Google Analytics to Hub
• [KIT-1240] - Cross Engine Visual Testing
• [KIT-1241] - Schema Enhancements1356] - UTF8 Characters not encoded correctly when an JSON object is inserted into the database

  
  

Other Improvements 

• [KIT-1249] - Logging system overhaul implementation (metrics , and trace)

Task

• [KIT-1060] - Use production build of React in non-dev environments- this should improve the performance and reliability of metric updates whilst opening the next stage of loggig and trace analysis. 
• [KIT-11751346] - Grant User Ability to pick Destination of Managed Graph and Template Publishing
• [KIT-1250] - Cluster RabbitMQ
• [KIT-1308] - Barebones React 16 Error BoundariesUpgrade SQL Alchemy and database libraries. SQL Alchemy has been upgrdaed to 1.25 and the underlying database libraries have also been updated.