Conversation
|
So far, it has been useful already as I was able to adjust the language and structure of ipfs/interface-go-ipfs-core#33 with 247bf38 |
|
Is this working? Should I try to run it? |
|
@jimpick It is working but it does nothing special yet. It is mostly bunch of scaffolding/setup code. I'm working my way through the testground codebase and learning how to yield IPFS daemons using the IPTB setup (#60) and/or direct in-proc nodes ipfs/kubo#6695. I'll change the state of the PR from |
|
Quick update: Still slaying dragons at -- ipfs/kubo#6695 --, I am super close though and I should be able to go back to writing the tests here today using that as a base. For reference, also learned the way that Netflix spawns their IPFS daemons, super clean https://github.com/Netflix/p2plab/blob/master/peer/peer.go |
77da95f to
396bd9f
Compare
002d8c5 to
d9d8f67
Compare
|
Getting the following panic when I run: Which comes from parsing the TOML manifest file -- https://github.com/ipfs/testground/blob/master/pkg/engine/engine.go#L228-L247 Scratching my head trying to figure out how is the manifest wrong. Tried using TOML validators including https://github.com/BurntSushi/toml and it parsed the manifest just fine. Any tips? |
|
@daviddias The Either |
|
aaaah! Epic facepalm! you are right. There is no such runner as local:exec yet. I completely missed that it was my param that was wrong and was looking for the bug in the wrong place. Thank you @Stebalien !! |
1c79f1d to
ffe666c
Compare
| addOptions := func(settings *coreopts.UnixfsAddSettings) error { | ||
| settings.Layout = coreopts.TrickleLayout | ||
| return nil | ||
| } |
There was a problem hiding this comment.
Thank you @hacdias! I wasn't figuring this goism (if it is a goism) out
There was a problem hiding this comment.
Reading this bit of code I see: "A func that takes the setting as argument and sets the option and then returns nil."
Can you explain to me if this is in fact idiomatic go and if it is, why sometimes use functional arguments such as https://github.com/ipfs/testground/blob/master/plans/dht/test/create-dht-node.go#L22-L25 and other times do like this?
There was a problem hiding this comment.
@daviddias I've used Go in the past but I've never actually seen something like this. As far as I understand, what we have here is a function that takes a setting as argument and sets the option and returns an error if it occurred.
Whoever wrong the code in first place is certainly more capable of explaining why they choose to do it this way than I am. And I'm also curious
There was a problem hiding this comment.
@hacdias thank you! @Stebalien could you shed some of wisdom here?
There was a problem hiding this comment.
@hacdias this is a pattern we frequently use in our code base https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis. It essentially hacks around Go not supporting optional parameters, by leveraging its support of variadic arguments.
The idea is that you call some function (e.g. ipfs.Unixfs().Add) with the parameters it requires (e.g. ctx), and optionally pass in any additional parameters via functions (e.g. ipfs.Unixfs().Add(ctx, option1, option2, ....). The error that you noticed is part of the function signature and as you can see no error is actually returned (we return nil). However, it's possible that sometimes we might want to return an error. For example, if we want to only set settings.A or settings.B, then if B is set after A we can throw an error. Another example is if inside the option we do some IO operation that might fail, then we can still handle the IO error. Check out that link for more info.
There was a problem hiding this comment.
@aschmahmann I get that blog post and I believed I understood it well when I wrote https://github.com/ipfs/testground/blob/master/plans/dht/test/create-dht-node.go#L22-L25. (Let's call this "Way A")
However, the way to get the trickle-dag option to Unixfs is way different in nature (Let's call it "Way B"), is is literally instead of passing the result of a func call (just like https://github.com/ipfs/testground/blob/master/plans/dht/test/create-dht-node.go#L22-L25 and what dave cheney describes) it takes in a function that received the config object to then change it.
Are both ways of doing this part of the same solution? If yes, why use Way B over Way A?
There was a problem hiding this comment.
@daviddias I'm not sure if I fully understood your point about the options being "different in nature", but yes they are both part of the same solution.
For "Way A" you used dhtopts.BucketSize(bucketSize). If you look at dhtopts.BucketSize https://github.com/libp2p/go-libp2p-kad-dht/blob/4b7d6fb886de322dd54d7c597e8ab163e087ad74/opts/options.go#L151 you'll see that it's a function that returns a function with the same signature as "Way B". In short this means "Way A" is "Way B" wrapped in another function for convenience and readability.
To make this even more explicit the code here should be replaced with coreopts.Unixfs.Layout(coreopts.TrickleLayout). We actually already have a "Way A" wrapper for the function created here 😄.
I'm not sure if we have an explicitly defined pattern for where/how we put these wrapper "Way A" functions. However, they're generally defined and searchable via code-completion or full-text searching the repo that the options are defined in.
|
@daviddias here, when you wrote:
|
I was just considering having the files at the directory leaves
Yes |
|
@daviddias agreed, having files scattered around every directory may not help with any debugging because we're just trying to make sure it can add a directory correctly! |
|
Is anyone aware how to add files to MFS using CoreAPI? Couldn't find any of the MFS commands. What about the CG too? /cc @daviddias @Stebalien |
License: MIT Signed-off-by: Henrique Dias <hacdias@gmail.com>
License: MIT Signed-off-by: Henrique Dias <hacdias@gmail.com>
License: MIT Signed-off-by: Henrique Dias <hacdias@gmail.com>
License: MIT Signed-off-by: Henrique Dias <hacdias@gmail.com>
License: MIT Signed-off-by: Henrique Dias <hacdias@gmail.com>
License: MIT Signed-off-by: Henrique Dias <hacdias@gmail.com>
|
@hacdias move to Review/QA when you want our review |
|
Thanks for the heads up @daviddias and yes, it's ready for review. Please note only the first two tests are fully implemented. All the others are missing implementation and tracking is being done on ipfs/interface-go-ipfs-core#93! |
daviddias
left a comment
There was a problem hiding this comment.
Some nits but overall LGTM. Once those nits are handled, feel free to click the "rebase and merge"
Co-Authored-By: David Dias <daviddias.p@gmail.com>
Co-Authored-By: David Dias <daviddias.p@gmail.com>
Co-Authored-By: David Dias <daviddias.p@gmail.com>
Co-Authored-By: David Dias <daviddias.p@gmail.com>
Co-Authored-By: David Dias <daviddias.p@gmail.com>
|
Are you sure about rebasing? This PR has 41 commits and its messages are not the best. I think it would be better to squash this ones |
|
squashed :) |

@daviddias when this issue was created:
This adds a new plan called Chew Large Datasets. Please read the README for further information about each test plan and how it works.
Tracking comment: #93 (comment)