Round trip

We need to fix round-trip from json-schema to effect schema and back for the implementation of anyOf.

This all started when I noticed that in tables and kanbans some information is being lost when we update the typename of the stored schema.

Broad context

ts dxos/packages/core/echo/echo-schema/src/json/json-schema.test.ts
ts dxos/packages/core/echo/echo-schema/src/json/json-schema.ts
ts dxos/packages/sdk/schema/src/projection.ts

...

/**
 * Composite of view and schema metadata for a property.
 */
export type FieldProjection = {
  field: FieldType;
  props: PropertyType;
};

/**
 * Wrapper for View that manages Field and Format updates.
 */
export class ViewProjection {
...

  /**
   * Get projection of View fields and JSON schema property annotations.
   */
  getFieldProjection = (fieldId: string): FieldProjection => {
...

    const { typename: referenceSchema } = getSchemaReference(jsonProperty) ?? {};
    const type = referenceSchema ? TypeEnum.Ref : (schemaType as TypeEnum);
    const format = referenceSchema
      ? FormatEnum.Ref
      : schemaFormat === FormatEnum.None
        ? typeToFormat[type]!
        : (schemaFormat as FormatEnum);

    const options =
      format === FormatEnum.SingleSelect && oneOf
        ? {
            options: oneOf.map((opt) => ({
              id: opt.const as string,
              title: opt.title ?? (opt.const as string),
              color: (opt as any).color,
            })),
          }
        : {};

    const values = {
      type,
      format,
      property: field.path as JsonProp,
      referenceSchema,
      referencePath: field.referencePath,
      ...options,
      ...rest,
    };

    const props = values.type ? this._decode(values) : values;
    log('getFieldProjection', { field, props });
    return { field, props };
  };

...

Zoom in context

// Excerpt from: const toEffectSchema
 } else if ('oneOf' in root) {
    if (root.oneOf?.every((schema) => 'const' in schema)) {
      const literals = root.oneOf.map((schema) => {
        const annotations = jsonSchemaFieldsToAnnotations(schema);
        const literal = S.Literal(schema.const);
        return Object.keys(annotations).length > 0 ? literal.pipe(S.annotations(annotations)) : literal;
      });
      result = S.Union(...literals);
    } else {
      result = S.Union(...root.oneOf!.map((v) => toEffectSchema(v, defs)));
    }

// Excerpt from: describe json-to-effect
 test('object with oneOf const values and annotations', () => {
    const jsonSchema: JsonSchemaType = {
      type: 'object',
      required: ['selectedOption'], // Add this if we want it required
      properties: {
        selectedOption: {
          oneOf: [
            { const: 'option-1-id', title: 'Small' },
            { const: 'option-2-id', title: 'Medium' },
            { const: 'option-3-id', title: 'Large' },
          ],
          type: 'string',
        },
      },
    };

    const schema = toEffectSchema(jsonSchema);
    const origSchema = S.Struct({
      selectedOption: S.Union(S.Literal('option-1-id'), S.Literal('option-2-id'), S.Literal('option-3-id')),
    });

    expect(prepareAstForCompare(schema.ast)).to.deep.eq(prepareAstForCompare(origSchema.ast));
  });

When we materialise the oneOf json schema to effect schema, it correctly converts the oneOf array into a Union type with Literal types for each const value. However, in practice we store a color and a label for each option!

Example!

"state": {
  "type": "string",
  "format": "single-select",
  "oneOf": [
    {
      "const": "b590d58c",
      "title": "Draft",
      "color": "indigo"
    },
    {
      "const": "8185fe74",
      "title": "Active",
      "color": "cyan"
    },
    {
      "const": "e8455752",
      "title": "Completed",
      "color": "emerald"
    }
  ],
  "title": "State"
}

When we convert the effect schema back to json it ends up being:

"state": {
  "enum": [
    "b590d58c",
    "8185fe74",
    "e8455752"
  ],
  "format": "single-select"
}

We're losing the color and title, + the higher level title too!

First, let's create a round trip unit test

test('single-select with oneOf round trip', () => {
  const originalSchema = {
    type: 'string',
    format: 'single-select',
    oneOf: [
      { const: 'b590d58c', title: 'Draft', color: 'indigo' },
      { const: '8185fe74', title: 'Active', color: 'cyan' },
      { const: 'e8455752', title: 'Completed', color: 'emerald' },
    ],
    title: 'State',
  } as any as JsonSchemaType;

  const effectSchema = toEffectSchema(originalSchema);
  const roundTrippedSchema = toJsonSchema(effectSchema);

  expect(roundTrippedSchema).to.deep.equal(originalSchema);
});

Some notes:

When we convert the json schema to effect schema, we correctly construct an enum of literals out of the option ids
We only want to perform this kind of conversion from enum -> oneOf for single select (and soon multi-select) formats
- We should be able to reuse the code that does this.
- Store title and color in annotations on the effect schema? These annotations could live in the single-select format file.
I think we will need to store the color and title information in annotations on each select option
I notice that when we rename the typename for tables, we lose all the titles there as well!
Maybe we should use the JsonSchema as the source of truth and not round-trip through effect schema, that way we could treat the effect schema as a non-isomorphic projection of the underlying JsonSchema.

ZaymonFC/schema-round-trip.md

Broad context

Zoom in context