This overloads the concept of a profile. A profile is meant to override values in a pre-defined configuration, and a task is a wholly new configuration. Is there an issue with a user supplying a separate configuration file when they want to run a migration?

I think this would be clearer if it was just:

I actually started with that design, but if we add more tasks in the future then we'll have a proliferation of config schemas, whereas by nesting it under the "task" key we only need to adapt that one schema. If you feel strongly about it I can change it back, but I thought this would lead to a more stable abstraction.

I don't think there's a difference, but I could be wrong. The way it is now, we have to check specific arguments for each task with a different schema as we add more tasks, and we dispatch on the "id" key to decide which schema to use. I don't think we can expect to know the precise arguments for each new task type before they are defined.

Instead, I'm proposing that we configure specific modules in each system in their own sub-map. For example, what if people want to run multiple tasks at once? What about multiple tasks on different sets of ledgers?

Unless I'm missing something, each new task will add a new schema. The only difference is whether that new schema is validated directly or if all of the "task" schemas are grouped into a multi schema dispatched on "id", but there will be at least one schema for each task either way.

These I think are the important parts of a task abstraction:

• no need to add new integrant handlers
• localized schema updates
• defined order of execution

I like nesting the task config underneath the task keyword because it lets us reuse the integrant handler for every task.

I think both approaches are rather localized for schema updates, though if you want to define multiple tasks we'd either need a vector of task-maps in the style that I wrote or a vector of task-ids in the style you wrote. Or we could just elide that entirely and say you can only run one task at time and that will force sequential execution.

I do think we should try to get the abstraction right so users don't have to learn multiple incompatible approaches to running.

I think we should have a separate key for the migration, and any other commands like this we'll run because the configuration is clearer. "reusing the integrant handler" doesn't make any sense to me. Each migration, or "task" as you put it, will have an entry point. Either there is a case statement that we'll have to modify each time we need to add something, or we define a new method for a new key. There is no difference.

These "tasks" are independent components in the system, so they should get independent configurations, with independent initializers.

Should I just rip out the whole task abstraction then? I think it's useful but if you don't we can do it your way.

I took it out, and I see what you mean - the difference is dispatching on ig/init-key vs fluree.server.task/run-task, so they're basically the same. No task abstraction means slightly more config but a shallower call stack.

I think the task abstraction actually had more config. With tasks, you had a task config which included an id along side the config for that id. The config is flatter how it is now, and you just have the config for each task that lives in its own key; no extra task key, no extra id key, and no need for an extra dispatch in the config parser.

I think we should do all validation up front, and not scatter malli throughout the codebase. This schema should live in the fluree.server.config namespace.

I think this file should have a different name now.

I think these configs would be more extensible going forward as a multi schema instead of ::or. The initial dispatch function could just be this:

(fn [scm]
  (if (contains? scm "sidMigration")
    ::sid-migration-config
    ::config))

We could modify that function (probably by replacing the if with a cond and adding more contains? checks) as we add more ways of running the system and their corresponding config types.

What does the multi-schema add that :or doesn't provide? I started doing this style of indirection before but I didn't want to invent a new name for regular runtime configs and the :or seemed to deliver all the functionality I needed without any additional fuss.

Functionality wise in this situation? Not much. I was more thinking about extensibility and clarifying intent, but I'm fine with it if you prefer it this way.

This is minor, but the resulting config map would be easier to deal with if it was flattened didn't have this extra ::sid-migragion key inside of it. I think this would be better:

And this is why I think my previous suggestion 👆🏾 would be clearer