diff --git a/.devrev/repo.yml b/.devrev/repo.yml
new file mode 100644
index 0000000..a1d97bf
--- /dev/null
+++ b/.devrev/repo.yml
@@ -0,0 +1 @@
+deployable: true
diff --git a/README.md b/README.md
index 72cbdde..2f2ff14 100644
--- a/README.md
+++ b/README.md
@@ -1,14 +1,15 @@
# chef-cli
-DevRev auxiliary CLI for ADaaS recipe development
+DevRev auxiliary CLI for Airdrop snap-in recipe development.
-General ADaaS documentation: https://developer.devrev.ai/snapin-development/adaas/overview
+General Airdrop snap-in documentation: https://developer.devrev.ai/airdrop
-chef-cli documentation is under [/docs](docs)
+chef-cli documentation: https://developer.devrev.ai/airdrop/initial-domain-mapping
## Install chef-cli
-A CLI tool is provided to assist you in this repo. Under [releases](https://github.com/devrev/adaas-chef-cli/releases), select the binary appropriate for your operating system, and install it in your path (or just remember its location). In the following steps we will assume it is available as `$ chef-cli`
+Under [releases](https://github.com/devrev/adaas-chef-cli/releases), select the binary appropriate for your operating system, and install it in your path (or remember its location).
+In the following steps we will assume it is available as `$ chef-cli`
### Install auto-completions
@@ -22,16 +23,9 @@ And restart your shell.
We support Bash and ZSH. Make sure to run the script from project home directory.
-To install PowerShell auto-completions, first run `Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass`. Then open the PowerShell profile (with `code $profile` or `notepad $profile`) and add this line (make sure to replace `/path/to/this/repo` with the path to this repository):
+To install PowerShell auto-completions, first run `Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass`.
+Then open the PowerShell profile (with `code $profile` or `notepad $profile`) and add this line (make sure to replace `/path/to/this/repo` with the path to this repository):
```text
/path/to/this/repo/autocomplete/chef-cli.ps1
```
-
-## Documentation
-
-- [Step-by-Step Guide](docs/step_by_step.md)
-- [Supported Types](docs/supported_types.md)
-- [Tips](docs/tips.md)
-- [Reasons why some mappings might be unavailable](docs/mapping_reasons.md)
-- [Using chef-cli as an MCP Server](docs/mcp.md)
diff --git a/docs/importing_articles.md b/docs/importing_articles.md
deleted file mode 100644
index 18cee1d..0000000
--- a/docs/importing_articles.md
+++ /dev/null
@@ -1,90 +0,0 @@
-# Handling Article Imports
-
-## Article mentions
-
-### Prerequisites
-Before tackling article mentions, you need to understand the [rich_text](./supported_types.md#rich_text) type.
-
-### What kind of mentions are supported?
-We support mentions to artifacts and articles. An inline attachment must be mapped to an artifact. A link to another article must be mapped to an article.
-
-### How to handle mentions?
-
-#### Inline attachments
-If an inline attachment is hosted in the source system, it must be created as an artifact in DevRev. The same link cannot be used as the attachment will be deleted in the source system when our customers deactivate the account. However, creating an artifact is not enough. The artifact must linked in the appropriate place in the article content.
-
-##### HTML
-```
-
-```
-
-##### Markdown
-```
-
-```
-
-##### Example
-Let's say the content of your external system looks like this:
-```
-This is an article with one image.
-
-```
-
-The content in DevRev should look like this:
-```
-This is an article with one image.
-
-```
-
-`don:core:dvrv-us-1:devo/0:artifact/1` is the ID of the artifact created in DevRev corresponding to the attachment with ID `29908544740244` in the external source system.
-To achieve this, you need to transform the content of the article to this:
-```
-[
- "This is an article with one image.
![](\"",)
"
- },
- "\" alt=\"download.jpeg\">
"
-]
-```
-The ref_type should be set to artifact and the ID should be the ID of the attachment in the external source system. The platform simply replaces the mention block with the ID of the corresponding artifact. The resolved value is not wrapped in double quotes.
-
-#### Links to other articles
-If there is a link to another article in the content of the article, you need to create a mention to the article. The link must be to an article that was either created in previous syncs or will be created in the current sync. At the extractor stage, it is impossible to predict the ID of the article that will be created in DevRev. So, this must be handled by the platform. This feature is only available for HTML format. However, since Markdown can contain HTML, you can use the same approach for Markdown as well.
-
-##### HTML
-```
-Contact our Support Team
-```
-
-##### Example
-Let's say the content of your external system looks like this:
-```
-You can create an account and log-in only with the company email.
-```
-
-The content in DevRev should look like this:
-```
-
You can create an account and log-in only with the company email.
-```
-
-`don:core:dvrv-us-1:devo/0:article/10` is the ID of the article created in DevRev corresponding to the article with ID `360059607772` in the external source system.
-To achieve this, you need to transform the content of the article to this:
-```
-[
- "You can create an account and log-in "
- },
- "\" target=\"_self\"> only with the company email."
-]
-```
-The ref_type should be set to the item type in the external system that is being mapped to articles. For example, if you're importing documents from the external system as articles, the `ref_type` should be set to documents. The ID should be the ID of the item in the external source system. The platform replaces the mention block with the ID of the corresponding article in DevRev as well as adds the href attribute with the appropriate value.
-
-## Managing Permissions
-
-You can manage permissions in the `shared_with` field. Permissions can reference users, groups and [platform groups](./platform_groups.md).
diff --git a/docs/initial_domain_mapping_setup.md b/docs/initial_domain_mapping_setup.md
deleted file mode 100644
index cb85504..0000000
--- a/docs/initial_domain_mapping_setup.md
+++ /dev/null
@@ -1,76 +0,0 @@
-# Chef-cli initial domain mapping setup
-
-## Prerequisites
-
-- Snap-in is set up, deployed, and activated
-- An import has been created and is in the mapping stage
-
-It's also recommended that you have the DevRev CLI authenticated appropriately. This can be done either with `devrev profiles authenticate -e prod -o -u ` or by running `make auth` in your snap-in repository.
-
-## Add your token as an environment variable
-
-Obtain a PAT-token from the Settings/Account tab of the devorg where you deploy your snap-in, and export is at DEVREV_TOKEN.
-
-You can also run the following command if you are authenticated with the CLI:
- ```bash
-export DEVREV_TOKEN=$(devrev profiles get-token access)
-```
-
-## Initialize the context of the sync
-
-To allow the cli to work in the context of that sync, you have to provide its identifying properties in an environment variable.
-The recommended method is to run:
-
-```bash
-chef-cli ctx switch --env prod
-```
-
-This will print the list of airdrop imports in the org. Select the one you like by running
-
-```bash
-eval $(chef-cli ctx switch --env prod --id ); chef-cli ctx show
-```
-
-If this method doesn't work, you can manually export the variable (replacing the values based on the logs of your running import):
-
-```bash
-export AIRDROP_CONTEXT='{"run_id":"1","mode":"initial","connection_id":"x","migration_unit_id":"0716","dev_org_id":"DEV-1kA79wWrRR","dev_user_id":"DEVU-1","source_id":"07-16","source_type":"ADaaS","source_unit_id":"x","source_unit_name":"x","import_slug":"x","snap_in_slug":"x"}'
-```
-
-Or you can use the interactive helper of the cli:
-
-```bash
-eval $(chef-cli ctx init); chef-cli ctx show > ctx.json
-```
-
-## Use the local UI to create a recipe blueprint for your initial import
-
-```bash
-chef-cli configure-mappings --env prod
-```
-
-If you are also creating DevRev -> external system sync, use:
-```bash
-$ chef-cli configure-mappings --env prod --reverse
-```
-
-Which enables mapping in both directions.
-
-If your org is not in US-East-1, you have to override an environment variable to make sure the tool reaches to the right server, eg:
-
-```bash
-ACTIVE_PARTITION=dvrv-in-1 chef-cli configure-mappings --env prod
-```
-
-where the options are:
-"dvrv-us-1"
-"dvrv-eu-1"
-"dvrv-in-1"
-"dvrv-de-1"
-
-The first function of the local UI is to assemble a 'blueprint' for concrete import running in the test-org, allowing the mapping to be tested out and evaluated.
-After it is used for the import, the mappings become immutable, but the chef-cli UI offers a button to make a draft clone, which can be edited again for refinements.
-
-## Continue to initial domain mapping
-
-You can now use the chef-cli web UI to create initial domain mappings.
diff --git a/docs/mapping_reasons.md b/docs/mapping_reasons.md
deleted file mode 100644
index 0556272..0000000
--- a/docs/mapping_reasons.md
+++ /dev/null
@@ -1,35 +0,0 @@
-# Reasons why some mappings might be unavailable
-
-There are several reasons why some mappings might be unavailable. We use transformation methods to map data from one format to another. These methods work on field scope and each have their own requirements. If the data does not meet these requirements, the mapping will not be available.
-
-1. A common reason is mismatch of types. For example, if a DevRev field is expected to be `rich_text`, but the field is set as `text` mapping to some fields will be unavailable. Refer to the [supported types](supported_types.md) section and the general DevRev documentation for more information.
-
-2. Only references can be mapped to references. Ensure that source system fields are correctly mapped to reference fields in DevRev.
-
-3. Currently, support for the `struct` type is limited. Marking a field as a struct in the metadata schema will make it unavailable for mapping outside of using the custom jq transformation method. Refer to the [metadata tips](tips.md#metadata-tips) for more information.
-
-4. Links are supported only on works and conversations.
-
-We provide the following transformation methods for custom fields:
-1. `make_constrained_simple_value` a simple method that along with mapping allows you to propagate validation constraints from the external system and enforces those in DevRev
-2. `make_enum` produces an enum field, where external field should be of type enum
-3. `make_uenum` produces an enum field, where external field should be of type enum
-4. `reference_custom_field` produces a reference field, where external field should be of type reference
-
-We provide the following transformation methods for stock fields:
-1. `make_custom_stages` makes custom stages in accordance with the stage diagram data provided in the domain metadata
-2. `map_enum` produces an enum field, where external field should be of type enum
-3. `map_roles` maps permission roles from external to DevRev format
-4. `use_as_array_value` produces an array field from a scalar (single-value) external field
-5. `use_devrev_record` enables use of a fixed reference to something in DevRev, where Don should be of right type
-6. `use_directly` is the identity operator, it returns exactly the input, where external field should be of the same type as the DevRev field. If external field is an array (is marked as collection) internal field has to be an array as well.
-7. `use_fixed_value` produces a fixed value in DevRev, which is useful to map required DevRev fields if source system doesn't have an equivalent field. Only available for boolean or enum DevRev fields
-
-We provide the following transformation methods for both custom and stock fields:
-1. `use_rich_text` produces a rich text field, where external field should be of type rich_text
-
-We provide the following transformation methods for constructed custom fields:
-1. `construct_text_field` produces a text field, where external field should be of type text
-
-
-`use_raw_jq` can be used on all fields. It enables the use of `jq` to transform data and should be used sparingly if no other transformation method is available.
diff --git a/docs/mcp.md b/docs/mcp.md
deleted file mode 100644
index d97ee2a..0000000
--- a/docs/mcp.md
+++ /dev/null
@@ -1,41 +0,0 @@
-
-[Model Context Protocol](https://docs.anthropic.com/en/docs/agents-and-tools/mcp) is an open protocol allowing LLMs to interact with applications.
-
-Chef-cli's `chef-cli mcp initial-mapping` subcommand implements an stdio-based MCP server, which provides tools for the AI to edit and test a local initial domain mapping file against a metadata file.
-
-To use it, you would need to configure your MCP client. In case of Cursor, this means putting
-
-```json
-{
- "mcpServers": {
- "airdrop": {
- "command": "chef-cli",
- "args": [
- "mcp initial-mapping"
- ]
- }
- }
-}
-```
-in `.cursor/mcp.json`, while for Claude Code, the same content goes to `.mcp.json` in the root of your project.
-
-
-To guide the AI in using the tool, we recommend providing it with a 'rule' or 'prompt' file along the lines of:
-
-```markdown
-When asked to perform airdrop initial mapping:
-
-- refer to the metadata.json for the external domain metadata.
-
-- use mappings.json as your initial domain mapping file.
-
-- Call 'use_mapping' to test out how the initial mapping behaves on the current metadata
-
-- Use MCP tools to manipulate the initial mapping when adding and removing record type mappings, or unmapping fields or mapping simple fields. For more complex field mappings you may edit the mapping file directly. Refer to the initial_mappings_schema.yaml for its proper format.
-
- If doing so, always use use_mapping to verify the mapping file is still valid.
-```
-
-You may need to modify the paths to the files you are directing the AI to.
-
-As of April 2025, MCP is an experimental capability of chef-cli, and it might or might not work well with your chosen AI model and MCP client, we however observed good results using Cursor 0.47 and Claude Code, where the AI was able to prepare an initial mappings, given the metadata, with considerable autonomy (though still with developer oversight).
\ No newline at end of file
diff --git a/docs/platform_groups.md b/docs/platform_groups.md
deleted file mode 100644
index 29ebf91..0000000
--- a/docs/platform_groups.md
+++ /dev/null
@@ -1,12 +0,0 @@
-# Mapping Platform Groups
-
-## Overview
-This document outlines the process for developers to map their external default groups to internal platform groups using the new mapping object. This functionality allows for flexible integration with the platform's permission system while allowing for re-mapping by the end user. You are able to map to groups created by default by the DevRev platform, for example to the *Dev users* and the *Rev users* groups.
-
-## Implementation Guide
-
-1. Developers need to create a mapping between their external default groups and the platform's internal default groups using a new mapping object.
- - To do this define a new object in external metadata.
- - The object should have one enum field with the possible values being the default groups available in external system.
-2. Developers can use IDM to map their object to the platform groups object in devrev (e.g.: to dev users and rev users groups).
-3. The platform groups object can be referenced in other objects, for example in the shared_with field of articles.
diff --git a/docs/step_by_step.md b/docs/step_by_step.md
deleted file mode 100644
index 201c7cc..0000000
--- a/docs/step_by_step.md
+++ /dev/null
@@ -1,464 +0,0 @@
-# Step by step guide
-
-## Define external domain metadata.
-
-The purpose of the metadata is to inform airdrop about the logical domain model of the external system, that is, to tell us what kind of _external_record_types_ are there, and what are their relationships, and what the type, human readable name, and other metadata of the fields are.
-
-Your extractor snap-in will be required to provide this metadata as an extracted json file with the item type of 'external_domain_metadata'. The format of this file is defined by the [following json-schema](../external_domain_metadata_schema.json).
-
-To check the metadata for internal consistency, you should use the following command after every step:
-
-`$ chef-cli validate-metadata < metadata.json`
-This will output any problems there may be with the metadata file.
-
-## Getting a good starting point metadata using the infer-metadata command
-
-`$ chef-cli infer-metadata example_data_directory > metadata.json`
-
-1. Collect example data from the external system, and place them in a directory. Each file should:
-
-- Contain the same type of records, named after their type.
-- Have .json or .jsonl extension, for example `issues.json`
-- Contain either a single json array of objects, or newline-separated objects.
-
-2. Run `$ chef-cli infer-metadata example_data_directory`, replacing example_data_directory with the relative path to this directory.
-
-3. Inspect the generated metadata, check if its field types are correct (see below on the detailed description of the supported field types), and consider the todo-s and suggestions the tool generates.
-
-Tips for best results:
-
-- It is recommended to provide 10-100 examples (but definitely not more than 1000) of each record type to get a good guess. (too few examples may result in not all relevant pattern being detected, too many examples may result in low performance).
-- The logically distinct fields of the record should be separate keys on the top-level.
-- It is ideal if example data is referentially consistent, allowing us to guess what field refers to what by comparing the sets of IDs. This means it is better to extract a complete but small set of data, instead of sampling randomly from a system with a lot of data.
-- The IDs should be strings, not numbers.
-
-This example metadata can be used to prototype initial domain mappings (by running a sync with it) and to generate example normalized data, but it is still just a guess: It has to be reviewed and refined.
-
-## Step-by-step approach to crafting the metadata declaration
-
-Since crafting metadata declaration in the form of an `external_domain_metadata.json` file can be a tedious process, a step-by-step
-approach is useful for understanding the metadata declarations and as a checklist to declare the metadata for an extraction from a specific external system.
-
-Metadata declarations include both _static declarations_, formulated by deduction and comparison of external domain system,
-and DevRev domain system and _dynamic declarations_ that are obtained during a snap-in run from external system APIs
-(since they are configurable in the external system and can be changed by the end user at any time, such as
-mandatory fields or custom fields).
-
-1. Declare the extracted record types
-
-_Record types_ are the types of records that has a well-defined schema you extract from or load to the external system, a domain object in the external system.
-
-If the snap-in is extracting issues and comments, a good starting point to declare record types in `external_domain_metadata.json` would be:
-
-```json
-{
- "record_types": {
- "issues": {},
- "comments": {}
- }
-}
-```
-
-Although the declaration of record types is arbitrary, they must match the `item_type` field in the artifacts you will upload.
-
-2. Declare the custom record types
-
-If the external system supports custom types, or custom variants of some base record type, and you want to airdrop those
-too, you have to declare them in the metadata at runtime. That is, the extractor will use APIs of the external system to
-dynamically discover what custom record types exist.
-
-The output of this process might look like this:
-
-```json
-{
- "record_types": {
- "issues_stock_epic": {},
- "issues_custom2321": {},
- "issues_custom2322": {},
- "comments": {}
- }
-}
-```
-
-3. Provide human-readable names to external record types
-
-Define human-readable names for the record types defined in your metadata file.
-
-```json
-{
- "record_types": {
- "issues_stock_epic": {
- "name": "Epic"
- },
- "issues_custom2321": {
- "name": "Incident report"
- },
- "issues_custom2322": {
- "name": "Problem"
- },
- "comments": {
- "name": "Comment"
- }
- }
-}
-```
-
-4. Categorize external record types.
-
-The metadata allows each external record type to be annotated with one category. The category key can be an arbitrary
-string, but it must match the categories declared under `record_type_categories`.
-
-Categories of external record types simplify mappings so that a mapping can be applied to a whole category of record types.
-Categories also provide a way how custom record types can be mapped.
-
-If the external system allows records to change the record type within the category, while still preserving identity,
-this can be defined by the `are_record_type_conversions_possible` field in the `record_type_categories` section. For example, if
-an issue that can be moved to become a problem in the external system.
-
-```json
-{
- "schema_version": "v0.2.0",
- "record_types": {
- "issues_stock_epic": {
- "name": "Epic",
- "category": "issue"
- },
- "issues_custom2321": {
- "name": "Incident report",
- "category": "issue"
- },
- "issues_custom2322": {
- "name": "Problem",
- "category": "issue"
- },
- "comments": {
- "name": "Comment"
- }
- },
- "record_type_categories": {
- "issue": {
- "are_record_type_conversions_possible": true
- }
- }
-}
-```
-
-5. Declare fields for each record type:
-
-Fields' keys must match what is actually found in the extracted data in the artifacts.
-
-The **supported types** are explained in-depth on the [supported types page](supported_types.md).
-
-If the external system supports custom fields, the set of custom fields in each record type you wish to extract must be
-declared too.
-
-Enum fields set of possible values can often be customizable. A good practice is to retrieve the set of possible values
-for all enum fields from the external system's APIs in each sync run.
-
-`ID` (primary key) of the record, `created_date`, and `modified_date` must not be declared.
-
-Example:
-
-```json
-{
- "schema_version": "v0.2.0",
- "record_types": {
- "issues_stock_epic": {
- "name": "Epic",
- "category": "issue",
- "fields": {
- "actual_close_date": {
- "name": "Closed at",
- "type": "timestamp"
- },
- "owner": {
- "is_required": true,
- "type": "reference",
- "reference": {
- "refers_to": {
- "#record:user": {}
- }
- }
- },
- "creator": {
- "is_required": true,
- "type": "reference",
- "reference": {
- "refers_to": {
- "#record:user": {}
- }
- }
- },
- "priority": {
- "name": "Priority",
- "is_required": true,
- "type": "enum",
- "enum": {
- "values": [
- {
- "key": "P-0",
- "name": "Super important"
- },
- {
- "key": "P-1"
- },
- {
- "key": "P-2",
- "is_deprecated": true
- }
- ]
- }
- },
- "target_close_date": {
- "type": "date"
- },
- "headline": {
- "name": "Headline",
- "is_required": true,
- "type": "text"
- }
- }
- }
- }
-}
-```
-Note, field keys are case-sensitive.
-
-6. Declare arrays
-
-If the field is array in the extracted data, it is still typed with the one of the supported types. Lists must be marked as a `collection`.
-
-```json
-{
- "name": "Assignees",
- "is_required": true,
- "type": "reference",
- "reference": {
- "refers_to": {
- "#category:agents": {}
- }
- },
- "collection": {
- "max_length": 5
- }
-}
-```
-
-External system fields that shouldn't be mapped in reverse should be marked as `is_read_only`. Depending on their purpose you can also mark fields as `is_indexed`, `is_identifier`, `is_filterable`, `is_write_only` etc. By default these will be set to false. You can find the full list of supported field attributes and their descriptions in the [metadata schema](../external_domain_metadata_schema.json#L160).
-
-7. Consider special references:
-
-- Some references have role of parent or child. This means that the child record doesn't make sense without its parent, for example a comment attached to a ticket. Assigning a `reference_type` helps Airdrop correctly handle such fields in case the end-user decides to filter some of the parent records out.
-
-- Sometimes the external system uses references besides the primary key of records, for example when referring to a case by serial number, or to a user by their email. To correctly resolve such references, they must be marked with 'by_field', which must be a field existing in that record type, marked 'is_identifier'. For example:
-
-```json
-{
- "schema_version": "v0.2.0",
- "record_types": {
- "users": {
- "fields": {
- "email": {
- "type": "text",
- "is_identifier": true
- }
- }
- },
- "comments": {
- "fields": {
- "user_email": {
- "type": "reference",
- "reference": {
- "refers_to": {
- "#record:users": {
- "by_field": "email"
- }
- }
- }
- }
- }
- }
- }
-}
-```
-
-8. Consider state transitions
-
-If an external record type has some concept of states, between which only certain transitions are possible, (eg to move to the 'resolved' status, an issue first has to be 'in_testing' and similar business rules), you can declare these in the metadata too.
-
-This will allow us to create a matching 'stage diagram' (a collection of stages and their permitted transitions) in DevRev, which will usually allow a much simpler import and a closer preservation of the external data than needing to map to DevRev's builtin (stock) stages.
-
-This is especially important if two-way sync will eventually be needed, as setting the transitions up correctly ensures that the transitions the record undergo in DevRev will be able to be replicated in the external system.
-
-To declare this in the metadata, ensure the status is represented in the extracted data as an enum field, and then declare the allowed transitions (which you might have to retrieve from an API at runtime, if they are also customized).
-
-```json
-{
- "fields": {
- "status": {
- "name": "Status",
- "is_required": true,
- "type": "enum",
- "enum": {
- "values": [
- {
- "key": "detected",
- "name": "Detected"
- },
- {
- "key": "mitigated",
- "name": "Mitigated"
- },
- {
- "key": "rca_ready",
- "name": "RCA Ready"
- },
- {
- "key": "archived",
- "name": "Archived"
- }
- ]
- }
- }
- },
- "stage_diagram": {
- "controlling_field": "status",
- "starting_stage": "detected",
- "all_transitions_allowed": false,
- "stages": {
- "detected": {
- "transitions_to": ["mitigated", "archived", "rca_ready"],
- "state": "new"
- },
- "mitigated": {
- "transitions_to": ["archived", "detected"],
- "state": "work_in_progress"
- },
- "rca_ready": {
- "transitions_to": ["archived"],
- "state": "work_in_progress"
- },
- "archived": {
- "transitions_to": [],
- "state": "completed"
- }
- },
- "states": {
- "new": {
- "name": "New"
- },
- "work_in_progress": {
- "name": "Work in Progress"
- },
- "completed": {
- "name": "Completed",
- "is_end_state": true
- }
- }
- }
-}
-```
-
-In the above example, the status field is declared the controlling field of the stage diagram, which then specifies the transitions for each stage.
-It is possible that the status field has no explicit transitions defined but one would still like to create a stage diagram in DevRev. In that case you should use set the `all_transitions_allowed` field to `true`, which will create a diagram where all the defined stages can transition to each other.
-The external system might have a way to categorize statuses (such as status categories in Jira). These can also be included in the diagram metadata (`states` in the example above) which will create them in DevRev and they can be referenced by the stages. This is entirely optional and in case the `states` field is not provided, default DevRev states will be used, those being `open`, `in_progress` and `closed`. If there is a way, the developer can categorize the stages to one of these three, or leave it up to the end user.
-The `starting_stage` field defines the starting stage of the diagram, in which all new instances of the object will be created. This data should always be provided if available, otherwise the starting stage will be selected alphabetically.
-
-In the current (v0.2.0) metadata format, it is no longer necessary to assign ordinal and stage_name to stages, the order and the human-readable name will be taken from the enum values defined on the controlling field.
-
-## Normalize data
-
-During the data extraction phase, the snap-in uploads batches of extracted items (the recommended batch size is 2000 items) formatted in JSONL
-(JSON Lines format), gzipped, and submitted as an artifact to S3Interact (with tooling from `@devrev/adaas-sdk`).
-
-Each artifact is submitted with an `item_type`, defining a separate domain object from the external system and matching the `record_type` in the provided metadata.
-Item types defined when uploading extracted data must validate the declarations in the metadata file.
-
-Extracted data must be normalized.
-
-- Null values: All fields without a value should either be omitted or set to null. For example, if an external system provides values such as "", -1 for missing values, those must be set to null.
-- Timestamps: Full-precision timestamps should be formatted as RFC3399 (`1972-03-29T22:04:47+01:00`), and dates should be just `2020-12-31`.
-- References: references must be strings, not numbers or objects.
-- Number fields must be valid JSON numbers (not strings)
-- Multiselect fields must be provided as an array (not CSV)
-
-Each line of the file contains an `id` and the optional `created_date` and `modified_date` fields in the beginning of the record.
-All other fields are contained within the `data` attribute.
-
-```json
-{
- "id": "2102e01F",
- "created_date": "1972-03-29T22:04:47+01:00",
- "modified_date": "1970-01-01T01:00:04+01:00",
- "data": {
- "actual_close_date": "1970-01-01T02:33:18+01:00",
- "creator": "b8",
- "owner": "A3A",
- "rca": null,
- "severity": "fatal",
- "summary": "Lorem ipsum"
- }
-}
-```
-
-Validate extracted artifacts using the following command (example for record type `issues`):
-
-```bash
-chef-cli validate-data -m external_domain_metadata.json -r issue < extractor_issues_2.json
-```
-
-You can also generate example data to show the format the data has to be normalized to, using:
-
-```bash
-echo '{}' | chef-cli fuzz-extracted -r issue -m external_domain_metadata.json > example_issues.json
-```
-
-## Deploy your snap-in in your test org, and run an import
-
-Relevant documentation can be found in the [DevRev documentation](https://developer.devrev.ai/public/snapin-development/locally-testing-snap-ins) and in [adaas-template repository](https://github.com/devrev/adaas-template).
-
-To deploy the snap-in, run `make auth` and `make deploy` in the snap-in repository. To activate the snap-in, run `devrev snap_in activate`. You can now create an import in the DevRev UI.
-
-The import will reach 'waiting for user input' stage. Wait there.
-
-## Complete the chef-cli initial domain mapping setup
-
-Next, continue with the steps outlined in [chef-cli setup](initial_domain_mapping_setup.md).
-
-When you are done you should have the chef-cli context set up and have the chef-cli local UI running in your browser.
-
-## Use the local UI to create initial domain mappings
-
-The final artifact of the recipe creation process is the `initial_domain_mapping.json`, which has to be embedded in the extractor.
-
-This mapping, unlike the recipe blueprint of a concrete import, can contain multiple options for each external record type from which the end-user might choose (for example allow 'task' from an external system to map either to issue or ticket in DevRev), and it can contain also mappings that apply to a record type category. When the user runs a new import, and the extractor reports in its metadata record types belonging to this category, that are not directly mapped in the initial domain mappings, the recipe manager will apply the per-category default to them.
-
-After the blueprint of the test import was completed, the 'install in this org' button takes you to the initial domain mapping creation screen, where you can 'merge' the blueprint to the existing initial mappings of the org.
-
-By repeating this process (run a new import, create a different configuration, merge to the initial mappings), you can create an initial mapping that contains multiple options for the user to choose from.
-
-Finally the Export button allows you to retrieve the `initial_domain_mapping.json`.
-
-## Tip: use local metadata in the local UI
-
-You can also provide a local metadata file to the command using the '-m' flag for example: `chef-cli configure-mappings --env prod -m metadata.json`, this enables to use:
-
-- raw jq transformations using an external field as input. (This is an experimental feature)
-
-- filling in example input data for trying out the transformation.
-
-In this case it is not validated that the local file is the same as the one submitted by the snap-in, this has to be ensured by you.
-
-## Test an import with initial mapping using the in-app UI.
-
-Once the initial mappings are prepared and, any new import in the org (with the same snap-in slug and import slug) where they are installed will use them. The end-users can influence the recipe blueprint that gets created for the sync unit trough the mapping screen in the UI, where they can make record-type filtering, mapping, fine grained filtering, low-code field and value mapping, and finally custom field filtering.
-
-Their decisions are constrained by the choices provided in the initial domain mappings. Currently the low-code UI offers limited insight into the mappings and their reasons, and in some cases, mismatches arise when something that worked in chef-cli doesn't offer the right options to the user, or not all fields that should be resolved are solved. To assist debugging such cases, chef-cli provides a command to extract the description of the low-code decisions that are asked in the UI. Please provide this to us when reporting an issue with how the end-user mapping UI behaves.
-
-```bash
-chef-cli low-code --env prod > low_code.json`
-```
-
-## Read metadata tips to improve the metadata
-
-See the [metadata tips](tips.md) section for more information on how to improve the metadata.
diff --git a/docs/supported_types.md b/docs/supported_types.md
deleted file mode 100644
index 72ca636..0000000
--- a/docs/supported_types.md
+++ /dev/null
@@ -1,62 +0,0 @@
-# Supported types
-
-Refer to the [metadata schema](../external_domain_metadata_schema.json) to help choose appropriate type for your fields.
-
-DevRev supports the following types:
-
-- `bool`
-
-- `int`
-
-- `float`
-
-- `text`
- - Text to be interpreted as plain text.
-
-- `rich_text`
- - Rich text is used in DevRev to represent text fields that can be formatted and can contain mentions. Eg.: description of an issue, body of a conversation, etc.
- - A simple rich text looks like one markdown string wrapped in an array: `["Hello **world**!"]`.
- Markdown should be compatible with [CommonMark Spec v0.30](https://spec.commonmark.org/0.30).
- - To support mentions `rich_text` can be formatted as an array of strings and mention objects like so:
- ```json
- [
- "Hello ",
- {"ref_type":"external_user_type", "id":"1...", "fallback_record_name": "John Smith"},
- "how are you?"
- ]
- ```
- - Mention represents any mention (user, issue, etc.) in rich text and is defined as:
- | Field | Type | Required | Description |
- | ---------------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------ |
- | `id` | String | Yes | Identifier of the item being mentioned. This could be a user ID or any other identifier, in the format used by the source system. |
- | `ref_type` | String | Yes | Type of the item being mentioned. Examples include "issue", "comment", etc. The recipe will convert this according to user mappings. |
- | `fallback_record_name` | String | No | The text to display if the mention cannot be resolved. This could be a user's display name or a ticket title, for instance. |
- - In reverse loaders should expect the following structure:
- ```json
- {
- "type": "rich_text",
- "content": [
- "Hello ",
- {
- "ref_type": "external_user_type",
- "id": "don:...",
- "fallback_record_name": "John Smith"
- },
- "how are you?"
- ]
- }
- ```
- - Articles support Markdown as well as HTML. For more details, refer to [article mentions](./importing_articles.md).
-
-- `reference`: IDs referring to another record. References have to declare what they can refer to,
- which can be one or more record types (`#record:`) or categories (`#category:`).
-
-- `enum`: A string from a predefined set of values with the optional human-readable names for each value.
-
-- `date`
-
-- `timestamp`
-
-- `struct`
-
-- `permission`: A special structure associating a reference to an user-like record type (the field `member_id`) with an enum value that can be interpreted as the role or permission level associated with that user. This is useful in a few cases when mapping fields with the same type in DevRev.
diff --git a/docs/tips.md b/docs/tips.md
deleted file mode 100644
index b084aef..0000000
--- a/docs/tips.md
+++ /dev/null
@@ -1,119 +0,0 @@
-# Tips
-## Metadata tips
-
-You are required to provide an external_domain_metadata file from your extractor, describing the logical schema of the external system.
-The detailed format of this metadata is defined by the [following json-schema](external_domain_metadata_schema.json).
-
-You can also find an [example](../demo/metadata.json).
-
-A few points about it:
-
-- The main purpose of the metadata is to define record types. Each record type should correspond to a homogenous set of records in the external system: a domain object that has a well-defined schema.
-
- In some cases this means simply declaring one record type for the api endpoints like '/comments' of the external system, but in other cases, external systems can have configurable custom types or subtypes (for example issuetypes in jira). In these cases the snap-in will need to query some API for the list of types, and produce a dynamic list of record_types in the metadata.
-
-- The record_types don't have hierarchy, each is a leaf type, corresponding to concrete records in files marked with that itemtype. Record type categories can be used to group them, this serves two purposes:
-
- 1. To be able to define mapping rules that apply to a dynamic set of record types, unknown at the time the snap-in is created
- 2. To tell the recipe system that a record can transition between two records types while preserving its identity.
-
-- The filed type 'int' is used to represent integer numeric values. In certain external systems identifiers of records or enum values are also stored as intergers. These are however not 'conceptually integers' in airdrops perpective.
- The natural format of integers is `null` | json numbers without decimals.
- Numbers encoded to strings (eq `"2112"`), or empty strings should not be used.
-
-- The primary key (id) of the record in the external system doesn't need to be declared as a field in the record type. Instead, id and created_date and modified_date has to be provided on the top level, and all other fields inside the data field. An example extracted record might look like this:
-
-```json
-{
- "id": "2102e01F",
- "operation": "created",
- "created_date": "1970-01-01T01:00:04+01:00",
- "modified_date": "1972-03-29T22:04:47+01:00",
- "data": {
- "actual_close_date": "1970-01-01T02:33:18+01:00",
- "created_date": "1970-01-01T01:08:25+01:00",
- "modified_date": "1970-01-01T01:00:08+01:00",
- "owner": "3A",
- "priority": "P1",
- "target_close_date": null,
- "title": "Something"
- }
-}
-```
-
-- All logical data types can be modified to be a collection instead.
- The natural format of a collection is a json array (or null), containing the natural format of its elements, for example:
-
- ```json
- {
- "reporter_ids": [
- {
- "ref_type": "user",
- "id": "2103232131",
- "fallback_record_name": "John Wick"
- },
- {
- "ref_type": "contact",
- "id": "2103232144",
- "fallback_record_name": "Lara Croft"
- }
- ],
- "tags": ["bug", "good-first-issue"]
- }
- ```
-
- Some systems provide collections that are enum values for references in a string, separated by some separator (for example comma or semicolon), eg:
-
- ```json
- {
- "reporter_ids": "2103232131,2103232144",
- "tags": "bug;good-first-issue"
- }
- ```
-
- This should be avoided, and the data normalized to the natural format in the extractor.
-
-- Structs are embedded json objects inside the given field. They are meant to represent data that consists of multiple elements naturally belonging together, for example a phone number or an address, but doesn't form its own record with identity.
-
- These are specifically helpful in case the whole struct is optional/nullable, but some of its fields are required.
- In this case the structs provide a cleaner representation that flattening it to the object containing it, and then applying some kind of conditional requiredness conditions.
- Example:
-
- ```json
- {
- "address": {
- "country": "US",
- "state": "TX",
- "city": "Austin",
- "address_line": "Rocket Road 1"
- },
- "phone_number": null
- }
- ```
-
- Many systems resolve references to embedded structs, for example:
-
- ```json
- {
- "creator": {
- "userId": "2103232144",
- "name": "Lara Croft",
- "role": "Adventurer",
- "email": "tomb@raider.com"
- }
- }
- ```
-
- Such 'detailed references' are not meant to be declared as structs in airdrop. They should be transformed either to be just a simple id reference, or a dynamically typed reference with exactly the fields id, fallback_record_name and ref_type.
- Therefore using references and structs inside structs is currently not supported.
-
-- In general, any fields in the metadata not otherwise mapped will be offered to the end user as custom fields, no additional effort needed.
-
-## Troubleshooting
-
-- Check if chef-cli version is up to date. You can find the version you are using with `chef-cli --version` and the latest available version under project releases.
-
-- You can find the logs for the snap-in package with the following command:
- ```bash
- devrev snap_in_package logs
- ```