Describe the bug
Some of benchmarks added in #3512 result in FAULTed VM. It happens because these benchmarks are old, they use NEWBUFFER with too large argument:
// PUSHINT32 1048575
// NEWBUFFER
Whereas since neo-project/neo-vm#514 VM limit for stackitem size is 2*maxUint16. The results corresponding with faulty benchmarks posted in #3512 (comment) are invalid.
The following benchmarks must be rewritten and fixed:
NeoIssue2723PoC_NewBufferPoC_CatPoC_LeftPoC_RightPoC_SubstrPoC_MemCpy
To Reproduce
-
Take the following code that runs benchmarks:
|
private static void Run(string poc) |
|
{ |
|
byte[] script = Convert.FromBase64String(poc); |
|
using ExecutionEngine engine = new(); |
|
engine.LoadScript(script); |
|
engine.Execute(); |
|
|
|
Debug.Assert(engine.State == VMState.HALT); |
|
} |
-
Replace Debug.Assert(engine.State == VMState.HALT); statement with the following:
if (engine.State != VMState.HALT)
{
throw new Exception(engine.State.ToString());
}
- See failing benchmarks, for example:
// Found 1 benchmarks:
// Benchmarks_Correct.PoC_Cat: DefaultJob
// **************************
// Benchmark: Benchmarks_Correct.PoC_Cat: DefaultJob
// *** Execute ***
// Launch: 1 / 1
// Execute: dotnet 7443c395-9aef-4a0e-806f-9e725e575501.dll --anonymousPipes 109 110 --benchmarkName Neo.VM.Benchmark.Benchmarks_Correct.PoC_Cat --job Default --benchmarkId 0 in /home/anna/Documents/GitProjects/neo-project/neo/benchmarks/Neo.VM.Benchmarks/bin/Release/net8.0/7443c395-9aef-4a0e-806f-9e725e575501/bin/Release/net8.0
// Failed to set up high priority (Permission denied). In order to run benchmarks with high priority, make sure you have the right permissions.
// BeforeAnythingElse
// Benchmark Process Environment Information:
// BenchmarkDotNet v0.13.12
// Runtime=.NET 8.0.3 (8.0.324.11423), X64 RyuJIT AVX2
// GC=Concurrent Workstation
// HardwareIntrinsics=AVX2,AES,BMI1,BMI2,FMA,LZCNT,PCLMUL,POPCNT VectorSize=256
// Job: DefaultJob
OverheadJitting 1: 1 op, 255355.00 ns, 255.3550 us/op
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.Exception: FAULT
at Neo.VM.Benchmark.Benchmarks_Correct.Run(String poc) in /home/anna/Documents/GitProjects/neo-project/neo/benchmarks/Neo.VM.Benchmarks/Benchmarks.Correct.cs:line 52
at Neo.VM.Benchmark.Benchmarks_Correct.PoC_Cat() in /home/anna/Documents/GitProjects/neo-project/neo/benchmarks/Neo.VM.Benchmarks/Benchmarks.Correct.cs:line 40
at BenchmarkDotNet.Autogenerated.Runnable_0.WorkloadActionNoUnroll(Int64 invokeCount) in /home/anna/Documents/GitProjects/neo-project/neo/benchmarks/Neo.VM.Benchmarks/bin/Release/net8.0/7443c395-9aef-4a0e-806f-9e725e575501/7443c395-9aef-4a0e-806f-9e725e575501.notcs:line 311
at BenchmarkDotNet.Engines.Engine.RunIteration(IterationData data)
at BenchmarkDotNet.Engines.EngineFactory.Jit(Engine engine, Int32 jitIndex, Int32 invokeCount, Int32 unrollFactor)
at BenchmarkDotNet.Engines.EngineFactory.CreateReadyToRun(EngineParameters engineParameters)
at BenchmarkDotNet.Autogenerated.Runnable_0.Run(IHost host, String benchmarkName) in /home/anna/Documents/GitProjects/neo-project/neo/benchmarks/Neo.VM.Benchmarks/bin/Release/net8.0/7443c395-9aef-4a0e-806f-9e725e575501/7443c395-9aef-4a0e-806f-9e725e575501.notcs:line 176
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
--- End of inner exception stack trace ---
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args) in /home/anna/Documents/GitProjects/neo-project/neo/benchmarks/Neo.VM.Benchmarks/bin/Release/net8.0/7443c395-9aef-4a0e-806f-9e725e575501/7443c395-9aef-4a0e-806f-9e725e575501.notcs:line 57
Expected behavior
All benchmarks must not use NEWBUFFER with invalid (too large) argument. All benchmarks should end in HALT VM state except those benchmarks that use endless cycles. Benchmarks with endless cycles must be checked against resulting execution error: the error must be caused by the end of GAS.
Platform:
Describe the bug
Some of benchmarks added in #3512 result in FAULTed VM. It happens because these benchmarks are old, they use
NEWBUFFERwith too large argument:Whereas since neo-project/neo-vm#514 VM limit for stackitem size is
2*maxUint16. The results corresponding with faulty benchmarks posted in #3512 (comment) are invalid.The following benchmarks must be rewritten and fixed:
NeoIssue2723PoC_NewBufferPoC_CatPoC_LeftPoC_RightPoC_SubstrPoC_MemCpyTo Reproduce
Take the following code that runs benchmarks:
neo/benchmarks/Neo.VM.Benchmarks/Benchmarks.POC.cs
Lines 387 to 395 in 5d2d8da
Replace
Debug.Assert(engine.State == VMState.HALT);statement with the following:Expected behavior
All benchmarks must not use
NEWBUFFERwith invalid (too large) argument. All benchmarks should end in HALT VM state except those benchmarks that use endless cycles. Benchmarks with endless cycles must be checked against resulting execution error: the error must be caused by the end of GAS.Platform: