We should surround the non-Python parts of this function with _Py_BEGIN_ALLOW_THREADS and _Py_END_ALLOW_THREADS to release the GIL. Similarly for the Assign function.

This can just be self.getloadavg = self.load_tracker.getloadavg

And since there are no later references to self.load_tracker, that doesn't need to be kept on the object (the references via the method will keep it from being deallocated)

Done, thanks :)

HANDLE_return_converter requires INVALID_HANDLE_VALUE for an error.

Good catch, thanks.

Python still supports Windows 7, which only allows a process to be assigned to one job at a time. If the OS version is prior to NT 6.2 (i.e. sys.getwindowsversion() < (6, 2)), use the creation flag
CREATE_BREAKAWAY_FROM_JOB (0x01000000) to try to break the child process out of the current job.

Aah thank you, I've been testing on Windows 10 and didn't read the documentation carefully enough, is there any particular reason to scope it down to <NT 6.2? Will there be a problem with just using CREATE_BREAKAWAY_FROM_JOB as the creationflags all the time?

There's no reason to break the child out of the job hierarchy in Windows 8+, especially since jobs are used more frequently (e.g. to implement silos, such as for Windows containers).

Also, trying to break away from the parent job complicates creating the process. I forgot to mention that Popen will fail with a PermissionError if the child isn't allowed to break away. This can be handled by retrying without CREATE_BREAKAWAY_FROM_JOB. In this case you could skip calling AssignProcessToJobObject. For example (untested):

CREATE_BREAKAWAY_FROM_JOB = 0x01000000

cflags = 0 if sys.getwindowsversion() >= (6, 2) else CREATE_BREAKAWAY_FROM_JOB
assign_to_job = True

try:
    self.p = subprocess.Popen(command, stdout=command_stdout,
                creationflags=cflags)
except PermissionError:
    if cflags == 0:
        raise
    self.p = subprocess.Popen(command, stdout=command_stdout)
    assign_to_job = False

if assign_to_job:
    _winapi.AssignProcessToJobObject(job_group, self.p._handle)

Call this in a try/except that handles failure either by ignoring it or with a warning. AssignProcessToJobObject will fail in Windows 7 if the parent process (i.e. the current Python process) is in a job that doesn't allow child processes to break away.

Thanks, I'll issue a warning since this to handle a rather rare case of the interpreter actually crashing in -jN mode.

Maybe this should reflect the WinAPI structure, i.e. the BasicLimitInformation field. Also, it would be nice to have PyWin32 interoperability, which does the latter and also uses dicts instead of simple namespaces.

Hmm, so I based this on CreateProcess which also uses attributes in a passed in object for this information. What do you mean by reflect the structure? Do you mean support all the fields for BasicLimitInformation?

For reference, here is how subprocess passes uses it with CreateProcess

I just used simple namespace because I was being lazy

Sometimes a dict is used, e.g. STARTUPINFO uses a dict for the new lpAttributeList support in 3.7. win32job uses dicts, but it's fine to stick with namespaces instead. I still would rather LimitFlags be set in a BasicLimitInformation attribute instead of at the top level. _winapi is low-level and undocumented, so it's best if it mirrors the actual API, if it's not overly awkward. I didn't meant to support all fields, however. _winapi gets extended only as required.

I would prefer to import the class there.

Done.

I dislike lambda. Would you mind to define a method here using "def" ?

I prefer lambda for this sort of short stuff but done

Please put the new code in a new file, like libregrtest/winloadavg.py.

Done

Please put the docstring inside the class.

Please don't use del() but add a close() method for example.

Hmm, are you sure about this? It would require us to keep a reference to load_tracker and it needs an:

if self.load_tracker:
    self.load_tracker.stop()

because we don't have a load tracker to stop on non windows systems.

What if the last line is incomplete: doesn't end with a newline character? Maybe you should put it back into a buffer, and concatenate it to the output, next time. Maybe use .splitlines(True) to check if there is a newline character?

I don't think its worth adding the extra complexity to handle this. Worst case is we miss a single point or two of data and the load number is slightly off.

Please add a comment to document the unit (seconds, no?).

It's possible that this function is only called every 5 minutes. I expect a lot of output in this case.

Why not running this function inside a thread?

This failure shows up when running in a thread: #8357 (comment)

Even at 5 minutes, that's 60 points of data. Even if the typeperf command puts out 100 bytes of output per line its not enough to saturate the buffer. And even if it does, those data points will get picked up eventually by the next call.

I seems like you spawn a subprocess even in test worker processes, whereas you wrote that it's useless. This code should be moved after handling slaveargs, no? Maybe move the slaveargs handling code earlier?

Not sure I follow, we already know if the runner is a slave based on the parsing of the arguments. The arguments have been parsed by this point, so this should be fine. I also tested and under task manager only one typeperf instance shows up under the main python process unlike before.