Simple MES

Simple MES is a web application designed to simplify the management of data sources for use with the Metadata Enrichment Service (MES). It is a Software as a Service (SaaS) offering hosted by RecordPoint, which eliminates the need for customers to create their own additional Azure resources.

Users can upload a spreadsheet or data file, and Simple MES provides a connection string that can be used in RecordPoint for MES integration. They can also develop more complex data solutions by connecting to data sources such as Snowflake.

Key Features of Simple MES:

• No Hosting Required: You don’t need to set up your own hosting environment. Instead, you can leverage RecordPoint’s world-class platform, which offers scalability, security, and resiliency.
• User-Friendly Setup: Simply sign in and start using without any installation. It integrates directly with Microsoft Entra for easy access management.
• Data Mapping: You can map both simple and complex data mapping scenarios, such as a simple spreadsheet or a more complex Snowflake data query.
• Supported File Types: Simple Mapping supports files in Excel (xls, xlsx), CSV, JSON, and Parquet formats.
• Access Control: Leverage Microsoft Entra permissions and conditional access.

How it works

Application Setup

Simple MES is designed to be user-friendly and doesn’t require installation, providing a hassle-free experience. It is a Software as a Service (SaaS) offering hosted by RecordPoint.

To setup Simple MES for your tenant, refer to Enabling SimpleMES.

After the application is added, you can access Simple MES using the same credentials you use to sign in to RecordPoint. This integration makes the user experience seamless and eliminates the need to remember multiple sets of login information.

How to Use Simple Mapping

MES is a RecordPoint feature that can add additional metadata to your items as they are ingested from your source to RecordPoint.

Simple Mapping is a powerful feature that enables users to leverage tabular data for the creation of a mapping table. This functionality is particularly useful in scenarios where you need to establish relationships between different data points.

To illustrate this concept, let us consider a practical example where you wish to assign a specific department and a designated document owner to a collection of files. This assignment is crucial as it plays a significant role in the subsequent processes of categorization and disposal of these files.

The mapping is performed in 3 steps:

1. Determine the attributes that will be needed to execute your business processes, such as categorization and disposal.
2. Identify a key that can be used to match the source source item to the mapping. For example a unique id or file path/name.
3. Create a structured table that represents the relationships between the key and their respective attributes.

Here is an example, that will stamp Department and Owner to any document that comes from SharePoint with the associated File Path:

1. Navigate to SimpleMES for your region.
2. Select + Add and enter a name for your new data source. This name is for reference and can be changed later.
3. Go to the Data tab, select Import, and choose your data file.
4. Once imported the fields will display.
5. Review the field data types. If any fields types are not automatically detected, select the correct type.
6. Designate the field that will act as the key to match this data with records in RecordPoint. There can be only one key, but you can change this field later if needed, as long as it is unique.
7. Click Connection Details. (It may take a moment as the system generates a new API Management Subscription.) Copy the API URL and API Secret.

Note: The API Secret can only be viewed once within SimpleMES as it stored using a one-way salted hash. Within the RecordPoint platform, this secret is stored securely and is encrypted at rest using an Azure Key Vault.

Behaviour of Fields

• Field names are shared across MES connections, this means if there is a clash between the field names then the first field to be created will take precedence. Unless the field is a shared field, then we recommend prefixing the field name with an identifier for all the fields in that project.
• When uploading the mapping file SimpleMES will attempt to convert numbers to a decimal or integer. If you require this to be a string then override the default mapping when uploading the excel/csv.
• If a field mapping for a particular item is empty (no value) by default this field will not be added to the item in RecordPoint.
• If a field value is removed from the mapping (null value) then this field will be removed from the record the next time MES processes this item.
• If a column is removed from the mapping then this field will be removed from all items the next time they are processed by MES.

Set Up the MES External Data Connection in RecordPoint

1. Create a new External Data Connection
   1. Navigate to Administration -> Configuration -> External Data
   2. Select New
2. General Settings
   • Title: Enter a name for the connection. For example, MES_SNOWFLAKE.
   • Description (optional): Provide a description to explain the purpose of the connection. Example: This MES integration allows the use of Snowflake to map data.
   • Enabled: Toggle the switch to enable the connection.
   • Update existing records with changes: Enable this option if you want changes to the enrichment metadata to update existing records. Please use a small connector if this flag set to true as it can spam our infra.
3. Connection Details
   • API URL: Enter the API endpoint for the connection
     Note: The URL cannot be modified after creation. If you need a different URL, you must create a new connection.
   • API Secret: Enter the API secret key {key to go here}
4. Content Source
   • Apply connection details to connector: Select the appropriate connector. Example: SharePoint Online Connector.
5. Source Primary Key
   1. For this example, you want to match the FileDirRef with the key in the Simple MES mapping.
6. Save the Connection
   • Select the Save button to finalize the setup.

RecordPoint synchronises MES data on a daily basis. Any alterations made to the mappings will take up to 24 hours to synchronise and be reflected in the MES.

Testing

1. Add a File: Upload a document to SharePoint (e.g., /sites/HR/Documents/Document1.docx).
2. Ingestion by RecordPoint: RecordPoint will ingest the file, which may take some time.
3. Check Metadata: Once ingested, open the file in RecordPoint. Look for the “Additional Information” section to see the metadata.

Additional Steps:

• For new connectors or locations, run Content Registration to ingest documents.
• For already ingested items, run a resubmit to add the additional metadata.