Data corruption due to case change

[bschick]

Describe the bug
During a patch call to change the value of the description field in the model below, the value of the credentialId field was lower-cased by ElectroDB. The value for credentialId is a base64Url string, meaning it was irrevocably corrupted by this bug.

A mistake on my part was not using the casing: 'none' setting for indexes (because I didn't know about it). This is easy to miss when starting with ElectoDB, and I think the default index case lowering behavior should be included in the quick start guide. Not using it for index values that may be unique only by case will otherwise cause problems (and it is hard to change after data has been written without going offline).

ElectroDB Version
electrodb@3.4.3

Entity/Service Definitions

export const Authenticators = new Entity(
   {
      model: {
         entity: "authenticator",
         version: "1",
         service: "quickcrypt"
      },
      attributes: {
         userId: {
            type: "string",
            required: true
         },
         credentialId: {
            type: "string",
            required: true
         },
         description: {
            type: "string",
            required: false
         },
         credentialPublicKey: {
            type: "string",
            required: true
         },
         credentialDeviceType: {
            type: "string",
            required: true
         },
         credentialBackedUp: {
            type: "boolean",
            required: false
         },
         transports: {
            type: "set",
            items: "string",
            default: () => [],
            required: false
         },
         userVerified: {
            type: "boolean",
            required: false
         },
         origin: {
            type: "string",
            required: true
         },
         aaguid: {
            type: "string",
            required: false
         },
         attestationObject: {
            type: "string",
            required: false
         },
         createdAt: {
            type: "number",
            default: () => Date.now(),
            readOnly: true
         },
         lastLogin: {
            type: "number",
            required: false
         }
      },
      indexes: {
         byUserId: {
            pk: {
               field: "pk",
               cast: "string",
               composite: ["userId"]
            },
            sk: {
               field: "sk",
               cast: "string",
               composite: ["credentialId"]
            }
         },
      }
   },
   {
      table: "Authenticators",
      client: client
   }
);

Repro steps

Value of sk (which is valid due to not using casing):

$authenticator_1#credentialid_ypkdnbah_1dsoa6frdibmaagju408tozbeljhs9qx78

 const patched = await Authenticators.patch({
    userId: id,
    credentialId: credId
 }).set({
    description: description
 }).go();

Note that the patch does not even modify the credentialId field, yet this is what changes.

Expected behavior
No case change of any field values.

Errors
Value of credentialId field before patch call:

YpKdnBAh_1dsoA6FrdIbmAaGJU408ToZBeljHs9Qx78

Value of credentialId field after patch call:

ypkdnbah_1dsoa6frdibmaagju408tozbeljhs9qx78

Additional context
For anyone else whodidn't using casing: 'none' when you should, I fixed the lack of casing by changing the table index definitions to include casing: 'none', then exported the table to S3 as JSON, ran the python script below on the exported JSON, uploaded back to S3, and then recreated the table by importing from S3.

Notes:

1. This only fixed the problem before the case change bug hits. If the value has already been corrupted you must retrieve from backup or some other source
2. This replaces all lower case string matches of the field value in the JSON...the script does not ensure it changes only key values. That would be easy to add, but I didn't need it.

#!/usr/bin/env python3

from pathlib import Path
import json
import sys

fields = sys.argv[1:]

directory_path = Path('.')
files = [f for f in directory_path.iterdir() if f.is_file() and not str(f).startswith('.') and not str(f).startswith('fixed-')]

for f in files:
    with open(f, 'r') as badFile:
        with open(f'fixed-{f}', 'w') as goodFile:
            for line in badFile:
                entry = json.loads(line)
                for goodField in fields:
                    goodValue = entry['Item'][goodField]['S']
                    badValue = goodValue.lower()
                    badField = goodField.lower()
                    line = line.replace(badValue, goodValue)
                    line = line.replace(badField, goodField)
                goodFile.write(line)

Called with:

../fix.py credentialId