How to add item to a dynamodb list with golang

add-item-to-list-dynamodb.go

/*
Took me a while to figure this out, this should never be so difficult IMHO. The dynamodb api could use more examples in its doc.

The documentation for doing this is scattered in a few places:

1. dynamodb api doc for golang: https://docs.aws.amazon.com/sdk-for-go/api/service/dynamodb/
2. the Update Expression: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.UpdateExpressions.html

The structs belowed are abbreviated for the sake of the demo: adding a string to the questions slice in the dynamodb table.

Note you will HAVE to use if_not_exists if using list_append(), and you can only use it with SET, otherwise it would return this error:
"the document path provided in the update expression is invalid for update", 
apparently it won't work if the attribute is empty or null initally.
*/

type Test struct {
	Id            int       `json:"id"`
	Name          string    `json:"name"`
	Questions     []string  `dynamodbav:"questions,omitempty"`
}

func (dq *DynamodbQuestion) Add2Test(testId string) bool {
	sess, err := session.NewSession(&aws.Config{Region: aws.String(config.AWS_REGION), Endpoint: aws.String(config.AWS_DYNAMODB_ENDPOINT)})
	svc := dynamodb.New(sess)
	if err != nil {
		fmt.Println(err)
		return false
	}
	av := &dynamodb.AttributeValue{
		S: aws.String(dq.Id),
	}
	var qids []*dynamodb.AttributeValue
	qids = append(qids, av)

	input := &dynamodb.UpdateItemInput{
		Key: map[string]*dynamodb.AttributeValue{
			"id": {
				N: aws.String(testId),
			},
			"uid": {
				S: aws.String(dq.UserId),
			},
		},
		ExpressionAttributeValues: map[string]*dynamodb.AttributeValue{
			":qid": {
				L: qids,
			},
			":empty_list": {
				L: []*dynamodb.AttributeValue{},
			},
		},
		ReturnValues:     aws.String("ALL_NEW"),
		UpdateExpression: aws.String("SET questions = list_append(if_not_exists(questions, :empty_list), :qid)"),
		TableName:        aws.String("tests"),
	}

Awesome ! They definitely need to improve the docs

thank you! but could you ever get it working with a list that is null initially?

@jackhwolf, it used to be not possible (dynamodb simply won't allow empty value), but recent changes seems to allow that finally: https://aws.amazon.com/about-aws/whats-new/2020/05/amazon-dynamodb-now-supports-empty-values-for-non-key-string-and-binary-attributes-in-dynamodb-tables/

It is also possible to use a struct to instead of manually specifying the attributes. See this post for details: https://runkiss.blogspot.com/2021/10/create-aws-dynamodb-using.html

you can only add elements to a list L with SET, but to add elements to a string set SS or number set NS you need to use ADD keyword.

this has the added benefit that you don't have problems with an empty list, it also prevents adding the same element to the list/string set multiple times

	input := dynamodb.UpdateItemInput{
		ExpressionAttributeValues: map[string]types.AttributeValue{
			":vals": &types.AttributeValueMemberSS{
				Value: []string{"Ferne"},
			},
		},
		UpdateExpression:         aws.String("ADD #ri :vals"),
		ExpressionAttributeNames: map[string]string{"#ri": "category"},
		// ---
		Key: map[string]types.AttributeValue{
			"PK": &types.AttributeValueMemberS{Value: "CATEGORIES"},
			"SK": &types.AttributeValueMemberS{Value: "CATEGORIES"},
		},
		TableName:              aws.String(ddbTableName),
		ReturnConsumedCapacity: types.ReturnConsumedCapacityTotal,
		ReturnValues:           types.ReturnValueNone,
	}

this is the go sdk v2, so might be slightly different for v1, but the idea is the same.

thx for posting this!

@jonny-rimek Thanks a lot for writing this up. I've been scratching my head on the exact issue you described. I tried your solution and found out it works quite well with ADD and SET, but it doesn't work with REMOVE. When I tried using REMOVE as below:

UpdateExpression: aws.String("REMOVE #ri :vals")

I got this error message: "operation error DynamoDB: UpdateItem, https response error StatusCode: 400, RequestID: AKGB0BQ349M5BNFL3U96IC677NVV4KQNSO5AEMVJF66Q9ASUAAJG, api error ValidationException: Invalid UpdateExpression: Syntax error; token: ":vals", near: "#ri :vals""

Have you tried using REMOVE on a set or list before? If so, did it work?

I'm also going to create an issue on github go sdk v2. If I hear anything back from aws team, I'll let you know here.

Thanks again for writing this up!

@jonny-rimek I read on The Dynamobook of Alexis DeBrie that one should use REMOVE to remove elements from a set. I just figured out that the keyword "DELETE" should have been used instead of "REMOVE"

@Shuo-Li glad I could help, and you solved your problem. REMOVE is for LISTS, which can contain both numbers and strings, DELETE is for SETS, which only contain either numbers or strings (and binary sets too afaik)

Unfortunately, the docs aren't super clear on that https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.UpdateExpressions.html#Expressions.UpdateExpressions.DELETE

@Shuo-Li lol completely unrelated but i used to work with Alex DeBrie at Hudl, great dude, what a pleasant surprise, thanks for the trip down memory lane..

@wliao008 I've learnt a great deal from Alex's book, which is probably THE best book about the single-table design with dynamodb database

@jonny-rimek The document link you posted was the one that helped me figure out DELETE should have been used. Wish I could have read it earlier.