Add a Joomla! Submit Testing Results Extension#12258
Add a Joomla! Submit Testing Results Extension#12258rdeutz wants to merge 2 commits intojoomla:stagingfrom
Conversation
## Situation today While we are in Alpha, Beta, RC development mode we don’t have a real overview how the packages are testet. We get some results when people submit issues but we don’t have an idea if people are testing in real live environments or if people are testing at all One of the problems is that there is no place someone can report a successful test result. This PR should solve this problem. ## Idea The idea is to have a specific extension for reporting test results. The date will be sent to the statistics server and we don’t ask people for personal data (they can submit personal data, I will explain the idea behind this later) It is as easy as possible, people ask simple question like „Is this a copy of a production site or a vanilla test site“, „Does the site run in a production like environment“, „Did you update or is it a clean install“ and so on. Just some easy questions. The next step is to report if someone tested a part of the CMS Yes with or without success or no. The testing result allow us to have a better understanding, what people are testing and where the package has problems. This will give us the chance to work on specific areas and improve the quality. The extension will only be enabled when we are in development mode (alpha, beta, rc) it doesn’t makes sense to have the extension enabled for a stable version of the CMS. I added some fields for personal data but these are not required, but if people give us there name we can list them as „PreReleaseTesters“ or if they give us the github account we can add points for testing so that people get some „fame“ back. # How to test * Apply patch * Make a discover install * play around If you hit send then you will see what would be send to the server. The endpoint isn’t developed at the moment, I though I will look first if people like the idea.
|
Does this need to be a part of core? What are we gaining having an API endpoint where sites can send a "this works" or "this does not work" (basically a boolean flag) and a duplicate of the stats server data object without additional details about what the user's doing? I honestly don't see much value in this as is. It's too generic and doesn't give any way for a user to explain what they are testing to validate a successful change or a way to report issues if they feel they've reached a fail state. |
| <license>GNU General Public License version 2 or later; see LICENSE.txt</license> | ||
| <authorEmail>admin@joomla.org</authorEmail> | ||
| <authorUrl>www.joomla.org</authorUrl> | ||
| <version>3.6.3</version> |
| <option value="3.6.2">3.6.0</option> | ||
| <option value="3.6.2">3.6.1</option> | ||
| <option value="3.6.2">3.6.2</option> | ||
| </field> |
There was a problem hiding this comment.
Is it sensible to have hard coded values. It will need constant updating
|
It might be generic at this state but it gives us information we don't have now. And we can make it more specific and add checklists for extensions to guide people to a process of testing. It is also a bit more then a boolean flag, you can say for each extension if you have tested it or not.
|
|
I have tested this item ✅ successfully on 413ba22 This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12258. |
|
Thanks for testing, the notice came from not proper testing if a var is set before using it. I am going to fix it later. |
|
Good idea @rdeutz ! 😃 |
|
I have my doubts that people will go find the extensions install it and then submit the results, if we don't ship it with core this all is a waste of time. Furthermore we only need to ship it while we are in alpha/beta/rc it don't has to be in a stable release. And even it is in a stable release, it can be disabled. |
|
Wouldnt this work the same as com_patchtester. People who want to contribute to testing will install the component and use it This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12258. |
We could use the same logic for including com_patchtester in those releases - which seems a good idea to me This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12258. |
|
@brianteeman I guess both this and com_patch_tester could somehow live in this branch for beta/rc. The only thing will be that people will have to do a discover installation, so the whole thing would remain simple (from db perspective) |
|
@DGT41 yes |
|
@rdeutz The libraries are listed in the unenabled extensions This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12258. |
|
|
||
| COM_JTEXTREPORT_DEFAULT="Joomla! Test Releases" | ||
| COM_JTESTREPORT_UNKNOWN_VERSION="Unknow Version" | ||
| COM_JTESTREPORT_ENABLED_EXTENSIONS="Enabled Extension" |
| COM_JTEXTREPORT_DEFAULT="Joomla! Test Releases" | ||
| COM_JTESTREPORT_UNKNOWN_VERSION="Unknow Version" | ||
| COM_JTESTREPORT_ENABLED_EXTENSIONS="Enabled Extension" | ||
| COM_JTESTREPORT_UNENABLED_EXTENSIONS="Un-Enabled Extension" |
| COM_JTESTREPORT_SITE_PERSONAL_INFO="Personal and Site information" | ||
|
|
||
| COM_JTEXTREPORT_FIELD_LIVESITE_LABEL="Type of Site:" | ||
| COM_JTEXTREPORT_FIELD_PRODENV_LABEL="Site runs in a production like env" |
There was a problem hiding this comment.
Site runs in a production like environment
(I think this needs some more explanation - I am not sure what you are expecting here)
| COM_JTEXTREPORT_FIELD_LIVESITE_LABEL="Type of Site:" | ||
| COM_JTEXTREPORT_FIELD_PRODENV_LABEL="Site runs in a production like env" | ||
| COM_JTEXTREPORT_FIELD_UPDATEFROM_LABEL="Site updated from:" | ||
| COM_JTEXTREPORT_FIELD_OVERALLRESULT_LABEL="Overall Test Result:" |
There was a problem hiding this comment.
(I think this needs some more explanation - I am not sure what you are expecting here)
|
We've kept the repo structure in a state that it only includes production assets (to the point that the Composer development dependencies like PHPUnit aren't included which makes maintaining that aspect of things more complex) with minimal extras (those extras basically being consolidated to the build and tests directories). I'm not convinced we should be littering the repo full of development only tools like this or patch tester, especially if don't even come installed by default (which makes packaging stable releases even more complex because you'd have to start stripping lines of code out of certain files). If it's in this repo it should be part of all packages and set up properly. |
|
What about only add link(-s) on backend (somewhere) and by one click user can install developer/tester component. Something like: Quick install one of the Joomla tester extensions. For example links can be visible only if error reporting is set to maximum or if joomla version contains some dev chars. |
|
Just use |
|
I like the idea! i could have google it to put the shame away from me but: Site updated from -> First one the language string is missing. Layout is broken at the bottom (Chrome) and following error on certain points: Is the list a list of the changes in the system? Would be perfect then to link to the specific Area (but i guess it´s too much effort to include this links) Later on on this page should be maybe an explanation what is happening with the data etc. |
|
Vanilla test is like vanilla ice cream. Clean and pure with nothing added
|
|
Let´s add this into the tooltip :-) |
|
Still not sure myself that this is something to add to the CMS myself On 28 October 2016 at 19:21, coolcat-creations notifications@github.com
Brian Teeman |
|
Setting to needs review. A decision needs to be made if this is something we want to have merged into the core - even if only distributed in staging and not in final releases This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12258. |
|
I have tested this item ✅ successfully on 413ba22 This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12258. |
Situation today
While we are in Alpha, Beta, RC development mode we don’t have a real overview how the packages are testet. We get some results when people submit issues but we don’t have an idea if people are testing in real live environments or if people are testing at all
One of the problems is that there is no place someone can report a successful test result. This PR should solve this problem.
Idea
The idea is to have a specific extension for reporting test results. The date will be sent to the statistics server and we don’t ask people for personal data (they can submit personal data, I will explain the idea behind this later)
It is as easy as possible, people ask simple question like „Is this a copy of a production site or a vanilla test site“, „Does the site run in a production like environment“, „Did you update or is it a clean install“ and so on. Just some easy questions. The next step is to report if someone tested a part of the CMS Yes with or without success or no.
The testing result allow us to have a better understanding, what people are testing and where the package has problems. This will give us the chance to work on specific areas and improve the quality.
The extension will only be enabled when we are in development mode (alpha, beta, rc) it doesn’t makes sense to have the extension enabled for a stable version of the CMS.
I added some fields for personal data but these are not required, but if people give us there name we can list them as „PreReleaseTesters“ or if they give us the github account we can add points for testing so that people get some „fame“ back.
How to test
If you hit send then you will see what would be send to the server. The endpoint isn’t developed at the moment, I though I will look first if people like the idea.