A new workflow for design work and feature requests #178
Labels
No labels
User research - Accessibility
User research - Blocked
User research - Community
User research - Config (instance)
User research - Errors
User research - Filters
User research - Future backlog
User research - Git workflow
User research - Labels
User research - Moderation
User research - Needs input
User research - Notifications/Dashboard
User research - Rendering
User research - Repo creation
User research - Repo units
User research - Security
User research - Settings (in-app)
No milestone
No project
No assignees
6 participants
Notifications
Due date
No due date set.
Reference
forgejo/discussions#178
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This lifts some aspects out of #156 that deserve more attention IMHO, and it addresses some recent discussions about where to put design discussions12.
Contrary to @earl-warren 's request, I do not think that the forgejo/discussions repository is an adequate place for design discussions. It's rather used for meta discussions about the project itself, long-term strategies and workflow adjustments. However, design decisions are a part of the development workflow. They are not code, but they are actual work that should live together with the rest of the development.
However, I acknowledge the need for developers to find tasks that are actionable and ready to be picked up. That's why I propose the following new workflow, especially for feature requests:
How the workflow can look in detail
Users are encouraged to describe the problem rather than the solution:: Feature requests typically consist of the description of a solution. However, often there are multiple solutions to one problem, and there are multiple problems with one solution. forgejo/forgejo#4119 is one such example, it is the combination of three different feature requests that all demanded different solutions for the same problem ("long scrolling to the README" in this example).
Often, feature requests want one more button to configure X, which is one step closer to hell (more settings, more logic, more edge cases, more complexity, more to learn for new users etc). And sometimes, the requested behaviour would be the best for everyone (as in forgejo/forgejo#3342 where the solution to configurable display of oauth sources is to simply specify a default login, which is likely easier to implement).
New feature requests are not ready to implement: They are investigated first, similar to how new bugs are triaged. This can either happen inside forgejo/forgejo (because I suspect that most feature requests will land there anyway) together with a label, or we create a separate place.
A research and design process is established: Similar to forgejo/forgejo#4119, we start an actual process for changes in Forgejo. This will put an additional step between user needs and development, but since the backlog of feature requests is growing anyway, I don't think this is an issue. On the contrary: Merging feature requests can reduce the backlog of open feature requests, and only the most valuable changes are implemented.
Fast-tracking in some cases? I'm wondering: Sometimes users have a very precise need, e.g. "integrate service X". I'm not yet sure if we should apply the process to every feature request, but I tend to say "yes". When a user requests an integration or very specific feature, we might still come up with a better idea. For example, when someone asks to integrate service X to do Z and someone else asks to integrate Y to do Z, we might do research and only implement one of both services to do Z, because it also covers the needs for the other user.
Separate or not to separate?: I'm very open to creating a Forgejo repository (= issue tracker) to focus on the design work if this is the general wish, but I'm also happy to continue working with labels. Since Forgejo doesn't make it easy to move issues, and since users are likely to continue generating feature requests in the forgejo repository, keeping them in one place sounds like the easier option. But … (see next)
Dealing with the open backlog? We can discuss how to deal with the backlog of existing feature requests. One option would be to treat issues with the enhancement label as "not ready" until they receive a corresponding classification by the User Research or UI team (since the latter also works closely on UX matters right now).
An alternative that might be appealing for developers who don't like the growing pile of open issues would be to
forgejo/forgejo#4140 (comment) ↩︎
forgejo/forgejo#4141 (comment) ↩︎
I have a very high motivation to act as a proxy between users and their needs and actionable feature requests for the Forgejo developers.
My time is constrained, but as I described, I don't think it's a big problem since there is not much more power to process feature requests anyway.
We could evaluate the strategy after some time and eventually scale up the User Research team or adjust the workflow if the feature requests are implemented quicker than new ones are researched :)
This sounds related to what I read yesterday: https://lobste.rs/s/ogxtsa/gnome_maintainers_here_s_how_keep_your#c_z3k0uc
Maybe a
maturity/*label:maturity/needs-refinement: the issue needs refinement before being worked on.maturity/documented-usecase: the user usecase is well documented. But it needs to be split into actionable tasks to be implementedmaturity/actionable-task: a PR can be openedCombined with a
benefit/*label could fit the bill:benefit/high: high impact/gainbenefit/medium: medium impact/gainbenefit/low: low impact/gainWhich means for instance that
benefit/high+maturity/actionable-taskissues can be worked on immediately.Whereas a
benefit/high+maturity/needs-refinementwill need more thinking, to get it todocumentedand thenactionable.All the
benefit/lowcan probably be initially ignored (except if they havematurity/actionable-taskandgood-first-issuefor newcomers to get started).I think a good approach is to:
Just to give an example, figuring out and implementing a sane workflow for release notes took me at least 40 hours in the past two months. And although it made good progress, it will likely take me another 50 to 100 hours before it is solid and automated. Another example is the backport workflow which required around 50 hours of work about three months ago. It still needs to be improved and would benefit from another 10 to 20 hours of work.
Triaging the issues and the feature requests is similarly very time consuming and also needs tooling to be consistent over time.
If a method is designed before finding someone there is a very high probability that:
Does that make sense?
@oliverpool The
benefit/*label could be merged with theUser Research: Gain/*labels or the latter reused. They are pretty recent, only one issue was labelled in such a way.@earl-warren I volunteer to spend significant time on this, and this is my proposed workflow. I can go ahead and decide the remaining details. But I would like to receive feedback so that my workflow has a good interface with what the other developers expect and what makes their life easy.
The actual work for me will be to revisit the open feature requests and to find solutions that address several problems. I did this over the past months and I'm still doing it, and I think there is a big potential in this. I started with the issues that received the lowest activity (i.e.. that are stale for long time). It will get easier over time, because I had to construct a mental model of user needs to merge similar issues into one. The hard part is sometimes understanding what users want, especially in domains where I'm not very familiar with (e.g. the diversity of package formats).
Creating the issues in places where the others expect them or labelling them accordingly is the easy part IMHO. I agree that designing and refining the workflow will take some time, that's why I'd like to gather feedback first so that I can hopefully prevent entering some dead ends.
If there is agreement either for a workflow, or for letting me move ahead, I'll start extracting some low-hanging fruits first (there are some issues with plenty of information available already), putting some of them aside for further investigation, and then scheduling this together with another round of user research sessions to obtain the missing information to process the feature requests on hold.
Then I'm confident this will move in the right direction ❤️
The workflow you propose is fine by me. I will comment on your questions:
There will be exceptions. They can be discussed on a case by case basis and common sense should be enough to figure out the right way to deal with them. I would not worry about them in advance.
I think it is best to use a tracker different from https://codeberg.org/forgejo/forgejo/issues for multiple reasons:
I also think a different tracker should reference these issues, use them as a source of truth and not duplicate their content. This different tracker could also have tooling that updates the issues with information coming from this different tracker. For instance if 20 issues are in the scope of a user research effort that takes two months to complete, the tools could add a link to the user research recommendations to each of these issues. And label them as "there is user research associated with them" and help them climb the ladder to being implemented sooner rather than later.
Yes. A feature would be ready for implementation when they gain a set of labels that show they are. And a developer could then look for all features that reached maturity and expect to find everything they need to get to work. As opposed to engaging in an exploration that is necessary before the technical work to implement the feature can start.
I suspect this won't work and generate a lot of pushback. The "unquestionable gain" is going to be a matter of controversy. And needlessly so as it amounts to engaging in the preparation work to figure out if and how a feature is worth implementing and supporting.
@earl-warren I'm not sure if I can follow your points to the end.
I'm fine with using a separate issue tracker and only communicating back the results of the design and research process.
However, it seems like you recommend to keep the initial noise of feature requests by users in the forgejo/forgejo repository, which I do not understand. Wouldn't this mean that the repo remains being cluttered by issues in the process stage one, while stage two is separate and only stage three is returned to the repo?
I apologize if my next comment does not fully include your feedback.
From the feedback and concerns (a lot was also discussed in the Forgejo UI Matrix Chat), I think this is the most sensible approach for now:
Does this sound sensible to you? I think it will require the least issue movement possible, but only touch existing issues that are not actionable anyway.
It is a hunch.
Isolated feature request that are self-contained and can be discussed in isolation within a single issue can be managed in the Forgejo tracker, from the initial idea to completion. An example of that would be "Add nonce parameter in OIDC authentication code flows".
Feature requests that require larger discussions, groundwork, user research, planning and more such as federation or redesigning the UX and UI need to happen elsewhere. Otherwise they are going clutter the tracker. For instance it is good that Redesign: Notifications is in the user research tracker and it is potentially related to a number of notifications feature requests. How will they bound together? Are there different kinds of notifications that require different kind of backend & UX? This will, no doubt, lead to a number of discussions over a rather extended period of time. The conclusions of which will be that those features request that are found in the tracker will be updated. The update, such as merging two feature requests into one, raising the priority of another, rewording the description of others, will be linked to this larger context to explain why and how this is happening. But if the background work happens in the main Forgejo tracker, it will clutter it.
Users and potential contributors are not complaining about the current initial workflow and they provide feedback that is valuable. I would not change that: it works.
The problem is on what happens after this initial issue creation, how they are triaged and sorted.
This has been tried in the past and there was significant pushback. On reason is probably that closing a feature request is perceived as rejection, no matter how you word it. Another is that from the point of view of a user providing feedback (bug or feature), the conversation should be about what they want to discuss and not about how Forgejo is structured and the workflows that are in place.
In each feature request there is a "Needs & benefits" section. Rather than a single binary label (gain) they should reflect the maturity of this section. Using something like what @oliverpool suggested:
I did not notice you created labels already and the gain is not binary.
I think they are good as they are and that you began to assign them to issues 👍
Could you change the description to be about how to get such a label rather than about what this label will enable? What matters is that there is clarity on how and why such labels are assigned. It can be terse such as "Very high => there exists user research recommending this feature" or "Very low => concerns are being discussed".
What do you think?
I think that it would be quite helpful to move the "not-mature" issues to another repo (like "user-research" or "discussions"). The main reason for me would be the "noise":
If there was a possibility to get notification only for specific labels, it would be for me very helpful (but in the meantime, having the discussion happen in another repo would be a good enough solution).
Side idea: allow multiple "Issues" tab per repo, with some filter based on the tag:
Bugs,Discussions...However my main concern would be: when should a discussion happen in
discussions, when should it happen inuser-research? (since theforgejorepo would be dedicated to actionable/mature issues).I would suggest shortening the length of the label (I find "User Research: Classified Gain: ..." quite long).
Another suggestion: keep only 3 levels, to make them easier to differentiate:
{benefit,gain}/high: this feature would be beneficial and there has been confirmation outside of the Forgejo tracker (e.g. forgejo/forgejo#923 (comment) where I pointed at 2 concrete examples where such feature would have helped){benefit,gain}/medium: there is consensus within Forgejo that this feature would be beneficial.{benefit,gain}/disputed(instead of{benefit,gain}/low), with a description like "there is currently no consensus regarding the benefit/gain of this topic". Those issues could be closed after a while.I'll try to rephrase the labels. They originate from #156 and did not yet take any of the current discussion into consideration.
I disagree to this. My work is to understand what users needs. But users don't explain their problem, they explain a solution.
Here is an example: forgejo/forgejo#4187
The users explain what they need: A setting. Settings are often bad, because they add to the complexity of software, they require testing all the cases, and overlooking a combination can lead to security issues or bugs.
The solution in this case is to alter the default workflow IMHO.
There are numerous cases like this (including the cases where I merged multiple feature requests into one). Users don't explain their problem, they explain the solution (because they often have something in mind).
The problem with the feature request template is that it asks for a solution. People cannot be neutral and explain their problem without thinking about a solution, and with a solution in mind, it's much harder to explain your actual problem.
There has been a long discussion a year ago on that topic. I think the original trigger of this discussion was precisely to require expressing a problem rather than a solution. And then a consensus was found that "Needs & Benefits" was a better way to achieve the same result. It has not been perfect but it has merits IMHO.
This is the nuclear option and it will considerably reduce the size of the issue tracker. It will also considerably reduce the input Forgejo gets. Maybe that's for the best and I'm too timid to dare suggest it is a viable option.
This comment was addressing that specific feature request as an example, it was not related to the workflow discussion here. Sorry that this was apparently unclear.
What about this: We update the template to explain to users what we expect in the "Needs & Benefits" section (e.g. encourage them to describe a problem). And we make the part about the actual feature optional, so that they don't have to provide a solution already.
The user research and design people will then try to do research and fill the empty suggestion.
Yes, great idea. And it may also be worth adding an optional "Problem description" in between and explain that if someone properly fills that section, it will give bonus points to the feature request. I don't think that possibility has ever been discussed. I suspect it will rarely be filled because everyone has a hard time understand what "explaining a problem" means while most everyone is inclined to think in terms of "I want that solution". But in some cases it will be filled and save valuable time.
It currently is
and could be
To be honest, I am sometimes a bit lost between
Needs and benefits,Feature DescriptionandScreenshots(andProblem description), I would propose:This way the author can:
Ideally the "Experience Report" can be expanded by other users, just like the implementation suggestion can be expanded by developers.
Taking forgejo/forgejo#4187 as an example:
@oliverpool I don't think that your proposal addresses the described problem that people think of a specific solution instead of describing the problem.
Your example includes the actual problem hidden in the Experience report. But the template would not really ask for the problem, only for the exact feature (which is one problem to the solution, but not the only or the best one).
I think it is more valuable to ask the user to explain its experience than describe the "problem". How would you formulate a "problem" from my example above? (issue 4187)
Besides a new feature might not address a problem (for instance a quality-of-life improvement enhancement does not necessarily have a problem). If there is an actual problem, then I should open a bug-report instead.
Moreover I feel that "Experience report" forces the user to describe something that they did experience (and not only imagined, as I have the feeling many feature requests are about).
In my opinion, a proper feature consists of three aspects:
Common feature requests include the first and the third element: I want to be able to control who is able to use push to create (what), by implementing a per-user setting (how).
As you can see, what and how are quite similar. Being able to control who can use push to create is closely tied to a per-user/org setting (as in your own example above, the "what" also describes it should be a per user setting, which is an implementation detail already. User research could indicate that another level of control could make sense, for example).
What is not addressed is the why: It would have been: I want to prevent inadvertently publishing my content.
The why can call for completely separate solutions, as discussed before. This is the "problem" I am talking about.
A problem can also be something like "I am confused that I always need to navigate here", and it could be turned into an enhancement to reorder elements. Problems are things that cost time, things that are confusing etc.
@oliverpool although you include the "Why", I suspect that users are already in a flow. Explanations about "why" are often incomplete and could just be things like "because I want to control who can use push-to-create", or "I want push-to-create to be only enabled when I need it", which is vague.
IMHO the why/problem/motivation needs to come first. Evaluating the feedback and coming up with a suggestion on how to improve the situation is a next, and often separate step.
The user who described the problem can participate in it and make or evaluate suggestions, but if they describe the feature first, you narrow everyone's mind on how the solution will look like - before even understanding the problem.
I totally agree. Hence the suggestion of “Experience report” instead of something usually incomplete like “why” or “problem”.
I think that “Experience report” nicely groups the “what” and “why”, which must be formulated as actual situations that the poster did have (whereas the “how” is an optional suggestion on how it could be solved).
Sounds reasonable, the ordering can be adjusted and simplified:
So the example would become:
Today I asked myself how good are the Needs and benefits section of the current ~400 feature requests? I went through the two dozen and here are my findings. Note that I did not factor in:
Includes the need and the benefit
The need or the benefit is missing
@oliverpool you wrote that you felt lost between all the fields presented in the template, which is an impression that I share. But are you confused by this particular section?
This makes sense and is the current template Needs and benefits is first, Feature description is next.
This particular comment prompted me to start looking at the current feature requests to understand what gave you this impression. What I observe is a majority of feature request where the "why" is present. Given that there is a lot of data, maybe it would be valuable to comb through it and see how effective / ineffective the current template really is?
@earl-warren I think we have a different understanding of why and what.
forgejo/forgejo#4185 as an example that you probably group in the "complete" category: It explains what they want, but not why. What is missing in the current renderer? We could implement the requested solution as-is, or we could dive in and learn more about different diagram types and their features. But this part is completely lacking: What did the user try to achieve? What is not supported in the current diagrams? I am looking for input like this.
forgejo/forgejo#3941 is an edge case, but still: It explains what the user wants. Looking into binary files is nice. But the motivation can be different: We might end up implementing hexdump (probably rather high effort to get it right and performant), but maybe the user just wants to differentiate certain binary files without ending to see if it's an image or an audio file. Maybe the need could have been addressed by calling
fileand printing the information likePNG image data, 1920 x 1080, 8-bit/color RGBA, non-interlaced, and maybe this has a better effort-gain-ratio. To find out, we'd need to know why they want to look into the binary data.I might be wrong in both cases. Maybe the user is 100% right and their proposed solution is indeed the best solution. But we cannot verify without deeper understanding.
I think we may have the same understanding but a different approach. When reading the Needs and benefits section, I look for the why / needs an benefit and I will make every effort to set aside the noise that may originate from inserting a what, poor English wording, confusion of concepts etc.
You will agree with me on the features request that I listed as not having one or the other, despite my best efforts to look for them 😄
I had the same first impression as you. But on second thought:
Now... it is entirely possible that the person is mistaken and that mermaid can already do all that. But this is a different matter.
I'm not saying these descriptions cannot be improved, my bar is low. I did not assert the quality, just the presence. And I think you will also be with me when I say that any kind of effort to improve the quality of these descriptions is going to be extraordinarily difficult. There presence alone is already quite an achievement.
@oliverpool @earl-warren I shortened the labels and adjusted the descriptions. They are now "Gain/".
However, when doing my work, I noticed a simple but (in retrospect) obvious problem: User research can never verify that a feature is unnecessary. You can observe 100 users that don't need something, but you can never be fully certain that something is not needed.
Given the low sample size of qualitative user research, I don't know how many interviews we'd have to conduct to consider all features that are not confirmed to be important as "not important". Every session shows a subset of needs, maybe 3 - 5 issues are important per user. However, there are hundreds of open feature requests that await confirmation.
The process to assign the priority will always include some guesswork. I'm thinking of also opening this up and define that the priority is determined either by user research or in collaboration with two Forgejo contributors. We could then do "issue scrubbing" where two or more contributors go through issues and decide between "keep or sweep".
Until we know this will work out, I propose something simpler
Never thought about it this way, it is like looking at the same problem under a different angle. 🤔
💯 Perfectly worded.
Here is an alternate idea that may be less work for contributors and less frustration for the person who opened the issue when it is ultimately closed. A comment is added to such issues that explains their feature request is currently at the bottom of the pile (low gain) and that they can help it move up in the world. The message can be copy pasted to all issues with low gain. If the author does not respond within a month, another message can be sent asking of they are still interested in this feature or some kind of gentle reminder in case they overlooked the first one. And if nothing happens a month later, the issue is closed.
To further avoid pushback on closing these issues in this way, the user research repository could provide a page that explains this methodology, together with a link to all those features request that are lingering because nobody (including their author) currently has time to care for them. Every six months or so a message could be broadcasted to call for help to help move these features up in the ladder of user research.
This will evidently end up into a an every growing pool of feature requests that were not followed up, like something you discuss between friends in a party. But it will be issues that are closed but not forgotten, waiting on the author to keep the conversation going.
@earl-warren I don't think your workflow is easier. At least for me, keeping track of things to come back later never really worked out. Going through issues with "low gain" and closing in case of agreement sounds much easier - at least for me.
I'll see to get something to reference before closing larger amounts of issues this way.
Another thing: I wanted to open a new issue, but maybe we can also reuse this thread, because it is very related:
I think we should try creating a forgejo/design repository. Design being wider than the visual appearance, it's about User Experience, workflows, efficiency, and even technical aspects could also become involved.
The Design repo is the interface between the UI and User Research Teams, they iterate on features, gather data, exchange ideas.
The workflow I have in mind:
What do you think?
CC @0ko and @Beowulf since you also collaborated with me on design discussions.
This is worth a try.
"low gain" is negative and combined with the closing of the issue will be unpleasant to the person who took the time to create this issue. Unnecessarily so, it could be replaced with "undefined gain" or "unclear gain" which is essentially the same but leaves the door open to improvement . The message closing the issue could be "We are unable to define the gain of this feature request at this point in time, if anyone has evidence (User Research etc.) to support it, please comment and it will be re-opened".
The "undefined gain" will be subjective in all cases, one or two contributors won't change much, IMHO. If a Forgejo contributor think that it is so I trust them. And if closing the issue triggers a reaction, the closing message explains what to do about it. And it will help avoid pushback that would not be productive.
Does that sound sensible?
This would be nice to have, currently we don't have anything related to design documented. I'm not sure if it will help much or if a team of 3 very part time members will be able to construct a design system.
What I value in my current workflow is very low overhead. Using Forgejo -> Documenting areas that need a look / Finding them in Issues -> Implementing improvements -> Documenting changes in a PR. It's efficient when my time is so fragmented, but it doesn't allow for large changes. Managing a separate repo to work on UI changes seems like a big overhead increase.
It's nice to use issues for iterating on design, but not effective when many iterations are needed and only 2 per day are possible.
I don't think this will be a good place to direct users to when they want to ask for an UI improvement. It's a separate repo and they won't have good understanding of how to communicate in here/what they're supposed to do to get someone implement something.
So, I think we need a repo, but we'll be figuring out what or what not to use it for after it is created. Currently it may not be possible to determine the workflows related to it.
@earl-warren I reworded the label to "Gain/Undefined". I think it's better now, less negative. It's a little ambiguous because unclassified issues are also "undefined" in a sense, but I think it's good as-is. Let's collect experience with the workflow and evaluate soon.
@0ko I think you misunderstand my goal. I want this repo to collect the design discussions we are already having. Earl Warren asked us to move it out of the main Forgejo issue tracker, and while I initially believed it is worth to show off our design work, it might indeed be better to create a new place which is tailored to our workflow requirements.
Things that are currently scattered but could be collected there:
I don't think users should be involved there, except if they really want to contribute. Considering this, I'd even think we might ask users to stay away from it when they generate too much noise.
Currently, I do have many cases where I'd like to link information I gather from user research and tiny interactions like Mastodon or issue comments to actual design ideas for new features, but I don't think there's a good place for this. I'd like to create one.
@fnetX I'd like to hear what you think of this comment I made. It is a little bit creative of me and I may have gone in a direction that's not quite what you had in mind.
Hmm, I wonder if it's best to assign labels without an immediate comment and only explain when we are actually closing something.
This saves effort on the one hand, and might invite less to discussions. In the example you linked, it is clear that this would be an improvement. It is an edge case, but it would still be good to have. If it was just a "nice to have" label, the user might be happy, although the issue would be around for a while. The "bottom of the ladder" phrase suggests it is not considered important, which is not exactly true. It is important, but there are many more important things.
I don't have a good solution with such edge cases (e.g. exotic setups and needs that meet together and have a really sub-optimal user experience, but there is no human power to even think about addressing them systematically)
BTW, I'm going to create the design repo soon. I also had the idea to close feature requests as being "included" in the design. Whenever a feature request has valuable feedback that is somehow transferred to some design, even if not yet implemented, the feature request in Forgejo can be included, because completing the design would solve them. Not sure if this makes sense. I might spell out the workflow I have in mind soon, though.
I think closing feature requests because they move up in the world is likely to be well received. But I've been wrong very often on this 😄 Let see what happens.
Sry, haven't seen this. I think a separate repo is good to maintain a better overview.
I would also like to ask what your thoughts are on using https://penpot.app/ for design, for example? Currently we always throw PNGs into the chat, but that has the big disadvantage that I can't easily build on someone else's idea, for example. Penpot is open source. If you are interested, I would like to test it on the summary tab.
Yes, feel free to play with penpot. Also take a look at this: https://discourse.opensourcedesign.net/t/a-small-tool-for-sketching-uis-wip-concept/3916
I wonder if we could integrate this into Forgejo so that design work gets part of code development natively.
(Context: I posted the concept on the opensourcedesign discourse )
Penpot: I think it is great for focussed design work and specification, it is not so helpful for ongoing "how might the UI look like" discussions, since it is a separate platform.
Yes, this is pretty much what it should be – quick creation of UI wireframes for specific discussions e.g. in chats or issue trackers.
It can't solve any of the high-level problems (what are the user needs, context of use etc.) though sometimes it can lead to interesting insights to sketch the "solution I want" (or to ask users to do so)
Based on the experiences from the past year, I have created a follow-up in #415 and appreciate your input there. Thank you very much!