Based on Tier Nolan's idea, I have made a way to atomically trade between BTC and NXT/NXTassets. I should be able to combine two such swaps so effectively a BTC <-> ALT (that supports p2sh) would be able to be done.
It took several attempts to solve this and there could well still be some edge cases, and this will be going into InstantDEX as a meta-exchange called "wallet". So best to find any flaws before it goes into beta test.
Here is the protocol:
0. Alice and Bob agree to the terms of the trade and share the pubkeys of their published address. Bob sends the rmd160(sha256(onetimepubkey)) to Alice along with a NXT release on secret linked to the same hash.
1. Both calculate the same p2sh address which allows redemption via 2of2 multisig or normal spend by Bob's onetimepubkey
2. Alice sends to Bob an unsigned funding transaction to the p2sh address for the amount agreed upon.
3. Bob creates a multisig refund tx for the entire amount back to Alice timelocked into the future, signs and sends to Alice.
4. Alice verifies the refund is proper and if it is, broadcasts the funding transaction, which creates a spendable output
5. Due to malleability, Bob needs to wait until it has enough confirms (1 should be enough?) for the value being traded. When he is satisfied, he spends the funds from the p2sh address using the normal path and this publishes his pubkey.
6. Alice submits this pubkey to release the NXT side of the transaction.
The above is the theory, in reality there are a few nasty surprises. If you use timelock and have a sequenceid of -1, timelock is ignored and it gets into the blockchain right away and the input is marked as spent immediately. If you have a sequenceid that is not -1, it is rejected with {"code":-26,"message":"64: non-final"}. So, Alice will have to keep in pocket the refund transaction and make sure to submit it if Bob hasnt revealed his pubkey.
This was from a manual run:
Alice addr:14XrJHhqMvDszMMwKR2d6u5PPvDeRieQoG -> 03b4143b6e7a064a78bc66dab0ca72251fea3d6fd8db1f4c48b91996d7af62dd05
Bob addr: 17Y5rr67RpTtuCd929dS6muyKZmjqJZQpr -> 037387677a21b8ebb472ff2d00892098dcd990a9a21b09b8e22aefece8b1cbbdc0
Bob's onetime addr 1NBKFDmmHSpSzFnP9bX5bDWnm9zqD6nUv4 -> 03c8a7e64c763e908549aa0f93974ab346a559b486eba32bf14338542b4282b916 -> e84e0808b8cd32c38fa2093a490efabc6be53d7a
Both can now calculate p2sh address of 37FsSwJ59z5UuRjNFjtfNYWVFquBEsUGHC from the redeemscript:
OP_IF OP_2 03b4143b6e7a064a78bc66dab0ca72251fea3d6fd8db1f4c48b91996d7af62dd05 037387677a21b8ebb472ff2d00892098dcd990a9a21b09b8e22aefece8b1cbbdc0 OP_2 OP_CHECKMULTISIG
OP_ELSE OP_DUP OP_HASH160 e84e0808b8cd32c38fa2093a490efabc6be53d7a OP_EQUALVERIFY OP_CHECKSIG
OP_ENDIF
I just combined a standard 2of2 and standard pubkeyhash into an conditional, but having never dealt with p2sh till this weekend, nor with signing of bitcoin tx, it was quite a lot of work to get this working as there doesnt seem to be any real support for custom p2sh scripts. Anyway, thanks to jgarzik's libbcoin, I was able to get the signing working in a day and stumbled about figuring out the "obvious" things about p2sh scripts.
[As an aside, if there is a way to verify that e84e0808b8cd32c38fa2093a490efabc6be53d7a is from the third pubkey of a 2of3 multisig address, then there would have been no need for any custom redeemscript. Just normal spends to a 2of3 multisig would have worked (with bob having 2 of the three addresses). Unfortunately I think it is crypto hard to go from msig address + pubkeyhash and 2 of 3 pubkeys to the third pubkey. And if that is not possible, then Alice has no way to know that bob didnt just give her a bogus pubkeyhash]
Bob sends to Alice: e84e0808b8cd32c38fa2093a490efabc6be53d7a
NXT tx: 1117487717739855000 hashed_secret: cc863b881f35bbeca374b25d018d4e17dc4aa19b Hash: RIPEMD160_SHA256
and Signed refund tx in pocket: 0100000001a9b3602a28960f1d855d1f52eea54f5e8b6fff469d54083ce433962b1c16dfe100000000f800483045022100d1439b9170a6d14e07dc6a11dd5c1aab80df302fe4f1de77e12c7859ba20361f02202dcc946b75995b62fb798b68e9bd443988c4004187f3c5cae52831b9415a712c014730440220754d43345edbeffd25cdfc6cbed16f59b1ef25c02092ff1752f0386e6a624e3d022075ef7578ccb2bbd0a8c58394ed7e4c5803b6ee66691623342fe19862146fda9f01514c6363522103b4143b6e7a064a78bc66dab0ca72251fea3d6fd8db1f4c48b91996d7af62dd0521037387677a21b8ebb472ff2d00892098dcd990a9a21b09b8e22aefece8b1cbbdc052ae6776a914e84e0808b8cd32c38fa2093a490efabc6be53d7a88ac68a510ec550130750000000000001976a91426be0216d0089626b3fc3a81f7d844f31a47e0fd88ac34b20500
then Alice broadcasts funding tx e1df161c2b9633e43c08549d46ff6f8b5e4fa5ee521f5d851d0f96282a60b3a9
Bob sees it on the blockchain and spends: 176026872b52b952c4649ee0b44026f87e2282aaaea7994117659a699662ae0a which reveals 03c8a7e64c763e908549aa0f93974ab346a559b486eba32bf14338542b4282b916.
Alice see's Bob's spend and submits an NXT phasing approval tx.
Prior to Alice funding the p2sh address, at worst some of Bob's temporary NXT reserves are held up for a time. After the trade is funded, we have a few cases:
a) Bob spends, Alice approves, both are happy
b) Bob doesnt spend to his address, then after timelock, Alice is able to submit the timelocked refund
c) edge cases: ?
clearly the timelocks need to be set properly, maybe 10 BTC blocks ahead and make it double the estimated time on the NXT side
All steps other than 5, can happen in seconds, so this is a BTC block timeframe in the mainstream case, 10 BTC blocks in the refund case. Not bad for a fully decentralized trade.
Once I verify a timelocked tx I am making can be submitted after the specified block, then I will fully integrate this into InstantDEX
James