File Read Adaptor

The File Read Adaptor reads the contents of a object based file.

Creating a File Read Adaptor

In order to use a File Read Adaptor it is best practice to create a file connection first. 

Create a new item via the Create menu item in the project editor, select the File Adaptor icon  and give the adaptor a name. 

Choose the connection from the drop down menu, and choose the file name and file contents destination

File Name: The File Name value can be provided by a literal, schema value or public variables

• For S3, this name should include any path outside of the top-level bucket.

File Contents Destination: The location for the body of the read file, and can be a schema value (recommended) or public variable

If the file contains a single JSON object this will be parsed and expanded at the location specified. If the file contains more than one JSON object or contains non-JSON data then the complete contents of the file will be saved as a string field at the location specified. Be wary of loading large files in this method as it is designed to load small objects and not large files. 

Currently only Amazon Web Services Simple Storage System (AWS S3) is supported, but it is planned to expand to other types (for example, Azure blob storage)

• AWS S3 is an object based storage scheme that presents a file-system view of the objects that it stores; folders are termed buckets and files are objects. For all intents and purposes it appears like a web hosted file system. The ability to read and write data from S3 buckets is useful for integrating with other systems that can only handle file-based integrations such as DMPs. 
• There is currently no way to list all of the files in a bucket - but this is something that we plan to add. A sample use case may be to have a stream of file names that require to be processed and the file reader processing those files. 

Validation Warnings