[CAP-68] Host function for getting executable for Address

[dmkozh]

This is a discussion thread for CAP-68 - Host function for getting executable for Address

CAP-68

This CAP provides a solution to the several issues encountered by the ecosystem developers:

• Axelar bridge implementation wanted to distinguish between the SAC and custom tokens in order to parse the token metadata differently
• Request for a mechanism for determining if a contract has been upgraded (Provide a mechanic to determine if a contract has been upgraded #1595, https://discord.com/channels/897514728459468821/1317258800377630730)

[leighmcculloch]

This is not stated in the CAP, but the get_address_executable will require the footprint to include the ledger entries that contain this data? I assume yes, just confirming.

Also if an address is given that has a ledger entry archived, will it panic rather than return None? I assume yes to this also.

[dmkozh]

This is not stated in the CAP, but the get_address_executable will require the footprint to include the ledger entries that contain this data? I assume yes, just confirming.

It is stated that the function behavior depends on the corresponding ledger entry, which implies that the entry has to be present in the footprint. I don't think the CAP should reiterate the footprint and state archival behavior every time - just specifying the ledger access is should be good enough.

Also if an address is given that has a ledger entry archived, will it panic rather than return None? I assume yes to this also.

This function behaves in the same fashion as any other ledger access function. The entry has to be present in the footprint in order to be accessible at all (otherwise the transaction fails on first out-of-footprint access). If there is an archived entry in the footprint, the transaction will fail before entering host (p22), but in p23 it should actually be auto-restored.

[leighmcculloch]

The motivation for the get_address_executable in the CAP is the idea of being able to pin interactions to specific executables. Since the CAP only adds getters, pinning will require getting the executable, comparing it to a value representing the expected executable, then invoking. The API presented may not do that as efficiently as it might appear.

This isn't too alarming to me, but just want to point out that this structure will be stored as a two-level nested Vec and so will cost a few host functions to construct, and it'll need to be constructed twice, the first to get back the value, the second to encode the expected value.

For example:

#[contracttype]
enum AddressExecutable {
    Contract(ContractExecutable),
    Account,
}

#[contracttype]
enum ContractExecutable {
    Wasm(BytesN<32>),
    StellarAsset,
}

Those types will be stored as:

Account:

Vec[Symbol("Account")]

Contract:

Vec[Symbol("Contract"), Vec[Symbol("StellarAsset")]]

Vec[Symbol("Contract"), Vec[Symbol("Wasm"), BytesN<32>(...)]]

I suggest we flatten the type, since the depth only emphasises the difference between accoutns and contracts, when that difference isn't meaningful here.

For example:

#[contracttype]
enum AddressExecutable {
    Account,
    Asset,
    Wasm(BytesN<32>),
}

Which will stored as:

Account:

Vec[Symbol("Account")]

Contract:

Vec[Symbol("Asset")]]

Vec[Symbol("Wasm"), BytesN<32>(...)]]

If the primary goal is to verify during invoking, we could also add an option to pass to invoking so that the type is only constructed guest side once.

[dmkozh]

I suggest we flatten the type, since the depth only emphasises the difference between accoutns and contracts, when that difference isn't meaningful here.

I wanted to reuse the existing ContractExecutable type here (we already use it in __check_auth). If we ever have more built-in contracts, then we'll need to just update the ContractExecutable enum once for everything. I'm not sure if marginal (1 u64 vector element) cost difference is a meaningful argument for or against the proposed data structure. I think data type consistency and reuse is generally beneficial and I'd like to reduce the number of new entities introduced. Another benefit is that contracts will be able to pin the ContractExecutable directly which will make the code that deals with it simpler (basically they don't need to deal with the weird Account case if they only care about the 'normal' contracts).

If the primary goal is to verify during invoking, we could also add an option to pass to invoking so that the type is only constructed guest side once.

I think this is too much work for a very little benefit - we'd need to introduce two new host functions for calling the contracts that aren't going to be used much and provide a really tiny optimization benefit. The getter will still need to be present in the interface because calling the contract is not the only use case for this.

[leighmcculloch]

We can maintain data type consistency by renaming the existing ContractExecutable to Executable, and adding account to the list. The __check_auth function can in that case just never use the account executable.

Agreed on not adding invoking, and doing so would be additive anyway and could be added at a later time if it was really necessary.

[dmkozh]

The __check_auth function can in that case just never use the account executable.

I think this is rather confusing for the implementers of __check_auth as they'll have an enum variant that is never used, but still needs to be handled in e.g. match statements.

I realize that the account/contract distinction is really annoying, but it does exist in the protocol and I find it hard to try hiding it in every context. Like we could in theory just return None/Err for accounts (and Err/None for the missing ledger entries respectively), but that would just make this function more restrictive and more annoying to use for no good reason.

[leighmcculloch]

I don't think the nesting is necessary if it can be as mentioned before a new flat type. Personally less nesting is easier to digest and the nesting introduces this strange dynamic about accounts vs contracts. So for me it's actually more about that signal we're sending to the ecosystem that these things are more different than we want them to believe.

Editing two enums also doesn't seem that bad, or one where the value doesn't exist sometimes (today at least).

But I agree with you that from a cost point-of-view, relative to the auth types, it has less depth and isn't that bad. 👍🏻

[dmkozh]

I think in the end the maintenance cost of two enums is not that high, and we can add conversion functions in the SDK if necessary. So I've flattened the enum: #1658

[tomerweller]

I think these should be split into two more aptly titled CAPs, one for the address executable getter and one for the string<>byte conversions. Lumping them into one aggregated CAP that references a protocol version seems opaque and likely won't age well

[dmkozh]

I'm honestly not sure if that provides a tangible benefit, but there is a downside of spending more time on the process. Do you think the split particularly helps with something here?

I'd like to move fast with the host functions when possible, especially when these follow the criteria outlined in this post (basically when the functions are trivial). We did that for the initial Soroban release (because creating 10 similar CAPs for different modules is really unproductive), and I'm not sure if it is actually problematic to continue bundling the host API improvements going forward. It would be great to continue iterating on the host API in each protocol release without much ceremony (especially given that we also have a lot of 'real' protocol work going on in parallel).

I think it could be even better if we could have short, 'to the point' docs for the trivial new host functions that can then be voted for together every protocol, but that requires a new process to be implemented.

[leighmcculloch]

I don't think separating these two into separate documents is that much of a time cost. The doc is largely a mishmash of sections where each sections belongs to each host function, so I think it's mostly a cut-and-paste half of the sections into another document.

+1 These host functions only need to be as long as they need to be, we don't need any ceremony, we just need the proposal and discussion.

[dmkozh]

Yeah, I understand that it's mostly copy-paste, but I'm not really a fan of copy-paste either. And there is also a burden of always distinguishing between the separate CAPs, which is rather annoying. But I do get the point that the section mixing is not nice for the reader; again, ideally we'd have per-function motivation/spec and no common CAP blurb at all. I guess I can separate these for now, but going forward I'll think on simplifying the whole 'adding the host functions' procedure.

[leighmcculloch]

I think optimising this flow is unnecessary. The cost is tiny. The process creates a collaboration point and helps uncover issues before implementation. Is it a bottleneck?

[tomerweller]

Thanks @dmkozh 🙏

[MonsieurNicolas]

Trying to understand better the stated use cases. Right now to me it seems to only really solve the "Is this a SAC?" use case reliably.

Are other ecosystem exposing this type of information like that in a way that is usable?

Are there example Soroban contracts for these? I am quite not sure how custom accounts would safely take advantage of this proposal.

In general contracts will be able to 'pin' the exact implementations of their dependencies
It's unclear to me if this is enough for really implementing "pinning": proxy contracts (or anything with dynamic dispatch) cannot be pinned with this mechanism, it sounds more like we need to enforce something based on actual call trees (similar to what we do for auth).

Custom accounts will be able to provide authorization policies based on the executables, for example, only allow a few vetted token implementations to be used.
similar to pinning, this does not work in the general case. I am not quite sure I understand the type of policies we're trying to implement (like: are we looking into enforcing certain pools too?).
I think here you're also looking at some sort of assertion on a (subset?) of the auth tree.

[dmkozh]

Are other ecosystem exposing this type of information like that in a way that is usable?

EVM has opcodes to get the both the hash and the bytecode itself for a given address (I don't think the latter is useful for Soroban today), which developers must be using to some degree.

Right now to me it seems to only really solve the "Is this a SAC?" use case reliably.

Even if that was the case, I don't see a good reason for not exposing a more generic function then just 'isSAC'.

It's unclear to me if this is enough for really implementing "pinning": proxy contracts (or anything with dynamic dispatch) cannot be pinned with this mechanism, it sounds more like we need to enforce something based on actual call trees (similar to what we do for auth).

If one wanted to make the proxy implementation 'pinnable', they could do so via exposing the executable of the underlying contract. I don't view the pinning as a general practice, rather as a design choice (i.e. one needs to consciously design for the dependency pinning), which is why I think restrictions like that are fine. I think @kalepail could comment more on the ideas around 'modular' contracts and specifically modular accounts.

If necessary, we could also provide a way to call contracts and restrict the set of executables to be called across the whole call tree. But I'm not sure if it's needed today and I also don't think it will be incompatible with this CAP.

similar to pinning, this does not work in the general case. I am not quite sure I understand the type of policies we're trying to implement (like: are we looking into enforcing certain pools too?).

Why not? Unlike the direct callers which in general only know what they're calling directly, custom accounts actually do know contract id of every contract that has requested auth. I think this should be sufficient most of the time.

In general, I think this function makes sense API-wise and can serve as a primitive for higher level patterns. Even if we eventually would need to implement something more complex, I don't think this function would need to be changed/deprecated, as it exposes pretty fundamental, standardized information about the addresses.

[leighmcculloch]

In general contracts will be able to 'pin' the exact implementations of their dependencies.

It's unclear to me if this is enough for really implementing "pinning": proxy contracts (or anything with dynamic dispatch) cannot be pinned with this mechanism, it sounds more like we need to enforce something based on actual call trees (similar to what we do for auth).

To illustrate, I think the sequence diagram below is something that the CAP sounds like it guarantees preventing, but it doesn't. At the least the CAP should say it only supports pinning direct dependencies.

But is pinning to direct dependencies a meaningful feature on its own? Unclear. No software package manager pins only direct dependencies.

custom accounts actually do know contract id of every contract that has requested auth

If you're writing a contract that needs guarantee over what wasms execute, I don't think its enough to only validate which wasms request auth. You might need the full call tree, not the auth call tree. This is an area that the auth call tree can be misleading because it can look like the execution call tree, when it isn't.

---

sequenceDiagram
    autonumber

    actor AA as Source<br/>Account
    
    participant H as Host
    participant CA as Contract A
    participant CB as Contract B
    participant CC as Contract C

    note over CA: wasm A1
    note over CB: wasm B1
    note over CC: wasm C1

    AA->>+H: invoke A
    H->>+CA: invoke

    CA->>+H: get wasm of B
    H->>-CA: B1
    CA->>+H: get wasm of C
    H->>-CA: C1
    note over CA: A feels good about<br/>continuing because<br/>B1 and C1<br/>are trusted

    CA->>+CB: invoke
    
    CB->>+CC: invoke
    CC->>+H: update_current_contract_wasm(C2)
    note right of CC: 😮 C changes<br/>its wasm<br/>unexpectedly
    H->>-CC: 

    CC->>-CB: return

    note over CC: wasm C2
    
    CB->>+CC: invoke
    CC->CC: unexpected behaviour
    CC->>-CB: return

    CB->>-CA: return

    note over CA: 😵 A thinks only<br/>trusted wasms<br/>have executed<br/>and has been fooled
    CA->>-H: return
    H->>-AA: return

Loading

---

I'm raising a concern I raised previously (#1595 (reply in thread)) then didn't pursue. When I raised it then, I thought the above scenario where a contract could change it's wasm effectively within a transaction wasn't possible.

[dmkozh]

To illustrate, I think the sequence diagram below is something that the CAP sounds like it guarantees preventing, but it doesn't.

I don't think the CAP claims to prevent this generically. But this could be avoided with more careful contract design where B pins Wasm of C (and A could be configuring that as well). If A trusts contract B's Wasm, then B's Wasm should either:

• not make cross-contract calls, so there is no concern about transitive deps
• not make cross-contract calls that meaningfully affect its state
• pin the cross-contract calls and follow the same principles recursively

I expect this to be used mostly with the first rule (i.e. no cross-contract calls). This way you can have modular contracts with pinned dependencies without worrying about the cross-contract call difficulties.

If you're writing a contract that needs guarantee over what wasms execute, I don't think its enough to only validate which wasms request auth.

I'm not sure I agree; if a contract doesn't require auth from you, then its logic must not be really relevant for you, so it shouldn't really matter what Wasm it uses. On the other hand, if you're concerned about the full internal state of the contract, then this CAP won't really help, but I don't think this is a reasonable goal to pursue at all.

[leighmcculloch]

I don't think the CAP claims to prevent this generically.

The CAP says:

stellar-protocol/core/cap-0068.md

Line 29 in 7fac1ab

- In general contracts will be able to 'pin' the exact implementations of their dependencies. While this is not necessary (or even desired) for majority of the protocols, there are use cases where pinning increases security. For example, this allows custom accounts to give restricted authorization privileges to the separate 'session' contracts without worrying about the change in the implementation of these contracts.

[leighmcculloch]

The CAP sets out to solve two problems noted at the top of the discussion:

1️⃣ Axelar bridge implementation wanted to distinguish between the SAC and custom tokens in order to parse the token metadata differently

2️⃣ Request for a mechanism for determining if a contract has been upgraded (Provide a mechanic to determine if a contract has been upgraded #1595, discord.com/channels/897514728459468821/1317258800377630730)

The security guarantees of 2️⃣ are imperfect I think, given #1626 (reply in thread).

If the change was only to be done for 1️⃣, maybe there are other ways to solve that problem. For example, if we dig into why folks are needing to identify the SAC, we could explore ways to remove that need so that contracts aren't distinguishing between the SAC and other tokens. If this has already been exhausted, no worries.

[leighmcculloch]

For example, if we dig into why folks are needing to identify the SAC, we could explore ways to remove that need so that contracts aren't distinguishing between the SAC and other tokens.

What I mean is identifying what the actual problem is. A while ago someone mentioned they were avoiding integrating with non-SAC tokens because the additional cost to execute them could pose a risk where the contract could become pegged / stuck. If the underlying problem can be solved, we might be able to enable usage of custom tokens not just the SAC.

[dmkozh]

What I mean is identifying what the actual problem is.

The SAC problem for cross-chain token wrapping is specifically about identifying SAC without hardcoding the contract ids or deriving the contract id from the metadata (and SAC identification itself helps distinguishing real SACs like XLM or USDC instances from impersonators). It's not about restricting the contract to only work with SAC.

[leighmcculloch]

distinguishing real SACs like XLM or USDC instances from impersonators

I understand an identification of XLM, in which case this could be a function to get the native contract ID, but how does an "if SAC" help identify an issued asset from impersonators? A contract ID can do that as effectively as an is SAC with a name check.

[dmkozh]

A contract ID can do that as effectively as an is SAC with a name check.

That requires understanding of how the assets map to the names and some XDR building/serialization, which is quite cumbersome dependency-wise and error-prone. It also just happens to be possible to do this way; executable getter is a more straightforward and semantically appropriate way. The same goes for native contract id getter - this seems pretty narrow and bespoke to me, as there is no concept of the 'native' token in the host itself.

[leighmcculloch]

That requires understanding of how the assets map to the names and some XDR building/serialization, which is quite cumbersome dependency-wise and error-prone.

The concept of finding a contract ID for an asset like XLM or USDC is something that all issuers and wallets are already exposed to in several places off-chain, so it's more odd imo to bring this on-chain when off-chain tooling already provides this capability.

$ stellar contract asset id --asset native
CDLZFC3SYJYDZT7K67VZ75HPJVIEUVNIXF47ZG2FB2RMQQVU2HHGCYSC

The case that isn't provided by off-chain is when a contract wants to broadly identify any SAC, not a specific SAC. And that's the case I want to drill into, because it suggests that contracts are excluding non-SAC tokens in general for some reason. One reason I've heard so far is the risk of pegging a contract due to higher execution cost of an associated token implementation, but I think we need to understand if that is the only reason or there are others.