Skip to content

feature/analyzers - symbolic shape checking #274

Closed
dsyme wants to merge 22 commits intofeature/symfrom
feature/analyzers
Closed

feature/analyzers - symbolic shape checking #274
dsyme wants to merge 22 commits intofeature/symfrom
feature/analyzers

Conversation

@dsyme
Copy link
Copy Markdown
Collaborator

@dsyme dsyme commented Feb 16, 2021 •
edited
Loading

A bunch of additional work over #207 to host shape checking tooling as an F# analyzer rather than as a standalone tool

Requires

Runs live checking as an F# analyzer

  1. Parallel build dotnet/fsharp branch feature/analyzers

    .\build -pack -c Debug
.\build -deploy -c Release

  2. Parallel build FSharp.Compiler.Portacode branch feature/analyzers

    dotnet build
dotnet publish

  3. Parallel build DiffSharp branch feature/merged which includes this branch

    dotnet build

Step by step manual testing

This builds on a stack of things in the pipeline so careful debugging is needed as things go awry. The following test the tech step by step

  • netfx fsc + Test.Analyzer.dll (two warnings should appear)
c:\GitHub\dsyme\fsharp\artifacts\bin\fsc\Debug\net472\fsc.exe --compilertool:C:\GitHub\dsyme\fsharp\artifacts\bin\Test.Analyzer\Debug\netstandard2.0 C:\misc\neutral-script.fsx
  • netcore fsc.exe + Test.Analyzer.dll (two warnings should appear)
dotnet c:\GitHub\dsyme\fsharp\artifacts\bin\fsc\Debug\net5.0\fsc.dll --compilertool:C:\GitHub\dsyme\fsharp\artifacts\bin\Test.Analyzer\Debug\netstandard2.0 C:\misc\neutral-script.fsx
  • netcore fsi.exe + Test.Analyzer.dll ('couldn't find stdin' should appear - correct because analyzer does this)
dotnet c:\GitHub\dsyme\fsharp\artifacts\bin\fsi\Debug\net5.0\fsi.dll --runanalyzers --compilertool:C:\GitHub\dsyme\fsharp\artifacts\bin\Test.Analyzer\Debug\netstandard2.0 C:\misc\neutral-script.fsx
  • netfx fsc + FSharp.Tools.LiveChecks.Analyzer sample (four errors should appear)
c:\GitHub\dsyme\fsharp\artifacts\bin\fsc\Debug\net472\fsc.exe --compilertool:e:\GitHub\dsyme\FSharp.Compiler.PortaCode\FSharp.Tools.LiveChecks.Analyzer\bin\Debug\netstandard2.0 e:\GitHub\dsyme\FSharp.Compiler.PortaCode\tests\FSharp.Tools.LiveChecks.Tests\test.fsx
  • netcore fsi + FSharp.Tools.LiveChecks.Analyzer sample
    - one error should appear (due to stop on first error)
    - note compiler too is via #compilertool in script
dotnet c:\GitHub\dsyme\fsharp\artifacts\bin\fsi\Debug\net5.0\fsi.dll  --runanalyzers  e:\GitHub\dsyme\FSharp.Compiler.PortaCode\tests\FSharp.Tools.LiveChecks.Tests\test.fsx
  • netcore fsi + FSharp.Tools.LiveChecks.Analyzer + DiffSharp ShapeCheck with --typecheckonly --runanalyzers
dotnet c:\GitHub\dsyme\fsharp\artifacts\bin\fsi\Debug\net5.0\fsi.dll --typecheckonly --runanalyzers E:\GitHub\dsyme\DiffSharp\examples\vae.fsx
  • Smoke test to put in script for execution while editing:
#compilertool @"e:\GitHub\dsyme\FSharp.Compiler.PortaCode\FSharp.Tools.LiveChecks.Analyzer\bin\Debug\netstandard2.0"
  • Eventually this to get a sample analyzer via the nuget package:
#r "nuget: FSharp.Tools.LiveChecks.Analyzer"

Requires branch feature/analyzers from dotnet/fsharp

  • the actual fslive.exe tool is run as a subsidiary process regardless of the environment of the tool the analyzer is running in. Inputs are binary-serialized back and forth
  • prototype is windows only
  • no recovery for subsidiary process
  • no adjustment is referenced DLLs for subsidiary process change
  • subsidiary processes may currently leak (cleanup on exit of host)

Using with DiffSHarp

c:\GitHub\dsyme\fsharp\artifacts\bin\fsc\Debug\net472\fsc.exe --compilertool:e:\GitHub\dsyme\FSharp.Compiler.PortaCode\FSharp.Tools.LiveChecks\bin\Debug\netstandard2.0\FSharp.Tools.LiveChecks.dll E:\GitHub\dsyme\DiffSharp\examples\vae.fsx

Won't be usable for a while, expect this to stay open a long time

@dsyme dsyme changed the title branch feature/analyzers - shape checking via analyzers feature/analyzers - shape checking via analyzers Feb 16, 2021
@codecov
Copy link
Copy Markdown

codecov bot commented Feb 16, 2021 •
edited
Loading

Codecov Report

❗ No coverage uploaded for pull request base (feature/sym@1920c8e). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@              Coverage Diff               @@
##             feature/sym     #274   +/-   ##
==============================================
  Coverage               ?   58.97%           
==============================================
  Files                  ?       36           
  Lines                  ?     7830           
  Branches               ?     1777           
==============================================
  Hits                   ?     4617           
  Misses                 ?     2330           
  Partials               ?      883

@dsyme dsyme mentioned this pull request Mar 11, 2021
Closed
@dsyme dsyme changed the title feature/analyzers - shape checking via analyzers feature/analyzers - symboilic shape checking Mar 11, 2021
@dsyme dsyme force-pushed the feature/sym branch 2 times, most recently from f96f999 to a5d2213 Compare March 11, 2021 16:13
@dsyme dsyme changed the title feature/analyzers - symboilic shape checking feature/analyzers - symbolic shape checking Mar 29, 2021
@dsyme dsyme mentioned this pull request Mar 29, 2021
Open
@pkese
Copy link
Copy Markdown
Contributor

pkese commented Apr 2, 2021

It might be interesting to extend this also with:

  1. some sort of checking of implicit conversion of dtype of tensor
    In Torch I sometimes get surprised by the fact that the algorithm is running slowly only to find out that I someplace multiplied a float32 tensor with a float64 and then Torch silently renders float64 computations.

  2. knowing which tensor is on which device
    This is a bit lesser problem in practice, because you get proper feedback in form of a runtime error, but it would nevertheless be a boon to see this while editing (similarly as we have analyzers for mismatching shapes even though that also pops up as runtime errors).

@dsyme
Copy link
Copy Markdown
Collaborator Author

dsyme commented Apr 3, 2021

Agreed, both are simple.

I think I will also extend this with a standalone command line tool to allow it all to be run without the F# analyser feature in place All with the aim of trialling it earlier

@dsyme dsyme closed this Mar 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dsyme @pkese