Consul Transaction API does not properly handle Service TaggedAddresses and Check Type fields.

[joelczou]

Overview

1. Consul Transaction API does not register the TaggedAddresses field in the AgentService struct that is sent from the client. Same with the Type field in the HealthCheck struct. It appears those fields are missed when converting from API objects to internal objects. https://github.com/hashicorp/consul/blob/v1.20.3/agent/txn_endpoint.go#L201

2. The Golang Consul API package fails to unmarshall the TxnResponse object when it contains a service with TaggedAddresses.
   It appears the ServiceResult in the internal TxnResponse object does not get converted to the API TxnResponse object properly https://github.com/hashicorp/consul/blob/v1.20.3/api/txn.go#L237.

Reproduction steps

From Unit Tests:

TestTxnEndpoint_UpdateCheck comparison fails when adding check Type to input and expected.
https://github.com/hashicorp/consul/blob/v1.20.3/agent/txn_endpoint_test.go#L521

TestTxnEndpoint_NodeService comparison fails when adding service TaggedAddresses to input and expected.
https://github.com/hashicorp/consul/blob/v1.20.3/agent/txn_endpoint_test.go#L691

TestAPI_ClientTxn fails to unmarshall data structure when adding service TaggedAddresses to the catalog registration.
https://github.com/hashicorp/consul/blob/v1.20.3/api/txn_test.go#L37

From HTTP API:

1. Create a Transaction API input file txn.json

[
{
    "Node": {
        "Verb": "set",
        "Node": {
            "Node": "n1",
            "Address": "192.168.0.10"
        }
    }
},
{
    "Service": {
        "Verb": "set",
        "Node": "n1",
        "Service": {
            "ID": "s1",
            "Service": "s1",
            "Address": "192.168.0.10",
            "TaggedAddresses": {
                "lan": {
                    "Address": "192.168.0.10",
                    "Port": 8080
                }
            },
            "Port": 8080
        }
        }
},
{
    "Check": {
        "Verb": "set",
        "Check": {
            "Node": "n1",
            "CheckID": "healthcheck",
            "Name": "healthcheck",
            "Status": "passing",
            "ServiceID": "s1",
            "ServiceName": "s1",
            "Definition": {
                "HTTP": "http://localhost:8080",
                "Interval": "10s"
            },
            "Type": "http"
        }
    }
}
]

2. Start Consul in Agent mode.

docker run -p 8500:8500 hashicorp/consul:1.20

3. Submit the transaction, returns success but Service TaggedAddresses and Check Type are not registered.

curl --request PUT --data @txn2.json http://127.0.0.1:8500/v1/txn

{
    "Results": [
        {
            "Node": {
                "ID": "",
                "Node": "n1",
                "Address": "192.168.0.10",
                "Datacenter": "dc1",
                "TaggedAddresses": null,
                "Meta": null,
                "CreateIndex": 18,
                "ModifyIndex": 18
            }
        },
        {
            "Service": {
                "ID": "s1",
                "Service": "s1",
                "Tags": null,
                "Address": "192.168.0.10",
                "Meta": null,
                "Port": 8080,
                "Weights": {
                    "Passing": 1,
                    "Warning": 1
                },
                "EnableTagOverride": false,
                "Proxy": {
                    "Mode": "",
                    "MeshGateway": {},
                    "Expose": {}
                },
                "Connect": {},
                "PeerName": "",
                "CreateIndex": 18,
                "ModifyIndex": 18
            }
        },
        {
            "Check": {
                "Node": "n1",
                "CheckID": "healthcheck",
                "Name": "healthcheck",
                "Status": "passing",
                "Notes": "",
                "Output": "",
                "ServiceID": "s1",
                "ServiceName": "s1",
                "ServiceTags": null,
                "Type": "",
                "Interval": "",
                "Timeout": "",
                "ExposedPort": 0,
                "Definition": {
                    "Interval": "10s",
                    "HTTP": "http://localhost:8080"
                },
                "CreateIndex": 18,
                "ModifyIndex": 18
            }
        }
    ],
    "Errors": null

[joelczou]

Hello, #22220 addressed the first issue, we can now register service TaggedAddresses and check Type fields when using the Consul HTTP API.

However the Golang Consul API library itself still doesn't work with the Transaction API as mentioned in the second point.

You can reproduce this by adding TaggedAddresses to the existing read transaction test case

consul/api/txn_test.go

Line 39 in d299ad0

Service: "foo",

, e.g.

			TaggedAddresses: map[string]ServiceAddress{
				"wlan": {
					Address: "2.2.2.2",
					Port:    12345,
				},
			},

Can also add a new write transaction test case

func TestAPI_ClientTxnWrite(t *testing.T) {
	t.Parallel()
	c, s := makeClient(t)
	defer s.Stop()

	s.WaitForSerfCheck(t)

	ops := []*TxnOp{
		{
			Node: &NodeTxnOp{
				Verb: NodeSet,
				Node: Node{
					Node:    "n1",
					Address: "1.1.1.1",
				},
			},
		},
		{
			Service: &ServiceTxnOp{
				Verb: ServiceSet,
				Node: "n1",
				Service: AgentService{
					ID:      "s1",
					Service: "s1",
					Address: "1.1.1.1",
					TaggedAddresses: map[string]ServiceAddress{
						"wlan": {
							Address: "1.1.1.1",
						},
					},
				},
			},
		},
		{
			Check: &CheckTxnOp{
				Verb: CheckSet,
				Check: HealthCheck{
					Node:      "n1",
					CheckID:   "healthCheck",
					ServiceID: "s1",
					Type:      "http",
				},
			},
		},
	}

	_, _, _, err := c.Txn().Txn(ops, &QueryOptions{})
	require.NoError(t, err)
}

Both of these will give the error json: cannot unmarshal object into Go struct field CatalogService.Results.Service.TaggedAddresses of type string

The root issue is that on the Consul server side, it return a structs.TxnResponse object which contains the Service result as a NodeService.

On the Golang client side, it tries to decode the TxnResponse as an api.TxnResponse which contain the Service result as a CatalogService.

The default JsonDecoder cannot decode the serialized NodeService into a CatalogService. This will need to be handled with custom deserialization logic.