Page Comparison

...

The File Connector currently only supports S3 as the Type (we plan to extend this to Azure Blob Storage soon). The Connector requires the S3 Bucket name which is the top-level bucket name without any additional path, and the AWS Access Key and Secret Access Key.

...

The File Connector provides a Test Connection button consistent with the other connection types. 

Writing a File to an S3 Bucket

In order to write an S3 file a filename, including path, and an object that represents the contents of the file: 

Image Added

The File Name value can be provided by a literal, schema value or public variable. The file is created on each write and will overwrite any existing file without creating an error. It is therefore suggested that a unique identifier is used to create the filename. Often buckets contain many thousands of small objects rather than one large file - most third party data loading processes are happy to process a large number of small files. 

The File Contents to Write can be any part of the schema. 

A JavaScript Node can be used to generate a file name that uses a timestamp and a unique identifier to ensure that files are ordered by creation date and also that there is no clash or accidental overwriting if multiple engines are processing and writing files simultaneously. 

Image Added

Reading a File From an S3 Bucket 

In order to read a file from an S3 bucket you need to provide a File Name and a File Contents Destination: 

Image Added

The File Name should include the path to file and filename. The Contents Destination is the location for the body of the file. 

If the file contains a single JSON object this will be parsed and expanded at the location specified. If the file contains more than one JSON object or contains non-JSON data then the complete contents of the file will be saved as a string field at the location specified. Be wary of loading large files in this method as it is designed to load small objects and not large files. 

There is currently no way to list all of the files in a bucket - but this is something that we plan to add. A sample use case may be to have a stream of file names that require to be processed and the file reader procesisng those files. 

Kitewheel Graph API Consumes XML

The Kitewheel Graph API will now consume valid XML posted to the graph API end point and convert this into a JSON object. The rules for conversion are as follows: 

• XML elements become JSON arrays
• XML attributes becomes "$" object with fields
• XML body text becomes "_" when there are attributes
• All element names are flattened to lowercase
• Attribute names are not flattened

<xml/>

{ "xml" : "" }

<xml>
  <element/>
</xml>

{ "xml":  {  "element": [ "" ]  } }

<xml>
  <element>a</element>
  <element>b</element>
  <element>c</element>
</xml>

{ "xml": { "element": [ "a", "b", "c" ] }

Front-End Performance

• React 16 
• Color chooser with better control
• Another chooser

...