Skip to content

Track method roots by precompile module#43793

Merged
timholy merged 1 commit intomasterfrom
teh/method_root_provenance
Jan 21, 2022
Merged

Track method roots by precompile module#43793
timholy merged 1 commit intomasterfrom
teh/method_root_provenance

Conversation

@timholy
Copy link
Copy Markdown
Member

@timholy timholy commented Jan 13, 2022

For roots of a method added during incremental compilation, track the
toplevel module build_id. This will ultimately allow such roots to be
"relocatable" in compressed IR, pending future changes in compression.

Part of #42016

@timholy
Track method roots by precompile module
161382d 
For roots of a method added during incremental compilation, track the
toplevel module build_id. This will ultimately allow such roots to be
"relocatable" in compressed IR, pending future changes in compression.
@timholy
Copy link
Copy Markdown
Member Author

timholy commented Jan 13, 2022

@nanosoldier runtests(ALL, vs = ":master")

@nanosoldier
Copy link
Copy Markdown
Collaborator

nanosoldier commented Jan 13, 2022

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

@timholy
Copy link
Copy Markdown
Member Author

timholy commented Jan 13, 2022

@nanosoldier runtests(["ADI", "AEMS", "AIBECS", "AMLPipelineBase", "ARDESPOT", "AbstractPlotting", "AgnosticBayesEnsemble", "AlgebraicRelations", "AutomotiveSimulator", "BED", "BasicInterpolators", "BasisFunctionExpansions", "BasisFunctions", "BenchmarkFunctions", "BigBed", "Binscatters", "Bio3DView", "BioRecordsProcessing", "CCDReduction", "CSVFiles", "CartesianGrids", "Catlab", "ChemistryFeaturization", "ChemometricsTools", "Clines", "ClusterEnsembles", "CombinatorialSpaces", "ConservationLawsParticles", "CopEnt", "CryptoCompare", "DataConvenience", "DataSkimmer", "DelayDiffEq", "DesignStructureMatrix", "DiffEqDevTools", "DiffinDiffsBase", "Dyn3d", "EDF", "EchelleCCFs", "EcologicalNetworksPlots", "EvoDynamics", "ExcelUtilities", "ExprOptimization", "FHIRClient", "FactorGraph", "FastAI", "Feather", "FeatureTransforms", "Flux3D", "ForecastData", "FormattedTables", "FunctionTabulations", "GEMPIC", "GenomicAnnotations", "GeoIP", "GeoStats", "GeoStatsDevTools", "GridapPETSc", "GroupedErrors", "GroupedTransforms", "GslibIO", "HOODESolver", "HetaSimulator", "HighFrequencyCovariance", "HomotopyContinuation", "HydrophoneCalibrations", "HyperbolicPlane", "ImageContrastAdjustment", "ImageEdgeDetection", "ImageSmooth", "ImplicitGlobalGrid", "IntegerSequences", "IntegerTriangles", "IntegratedOptics", "InvariantSetApproximation", "InvertibleNetworks", "IsopycnalSurfaces", "IteratedProcessSimulations", "JMcDM", "JudiLing", "JuliaInterpreter", "KCenters", "Khepri", "KmerAnalysis", "Knet", "KrigingModel", "LITS", "LSODA", "LassoPlot", "LinearCovarianceModels", "LocalSearchSolvers", "LogParser", "MCMCDiagnosticTools", "MCTS", "MCVI", "MLJClusteringInterface", "MLJEnsembles", "MLJLIBSVMInterface", "MLJModelInterface", "MLJNaiveBayesInterface", "MLJScientificTypes", "MLJXGBoostInterface", "MPIFiles", "MPIMeasurements", "MakieTeX", "ManuscriptOrthography", "MarketData", "MathematicalSystems", "MathepiaModels", "MatrixProfile", "Mazes", "McCormick", "MemPool", "Metida", "MetidaCu", "MetidaNLopt", "MetidaStats", "MixedModelsExtras", "MonthlyDates", "MotifRecognition", "MultiwayNumberPartitioning", "NCBITaxonomy", "NMRTools", "NoiseRobustDifferentiation", "NonNegLeastSquaresMLJInterface", "Nonconvex", "NonconvexJuniper", "ODE", "ODEInterface", "ONNXNaiveNASflux", "OhMyArtifacts", "Omniscape", "OpenQuantumTools", "OptimKit", "OptimalTransmissionRouting", "OrdinalGWAS", "POMDPPolicies", "PPLM", "PQPolygonSDK", "ParallelAnalysis", "ParameterEstimateScatterPlots", "ParameterizedFunctions", "ParticleScattering", "PerceptualColourMaps", "PerronFrobenius", "PersistenceDiagramsBase", "Phylo", "PlotShapefiles", "PlotlyJS", "Porta", "PossibilisticArithmetic", "PowerModelsAnnex", "PowerModelsDistribution", "PowerModelsInterface", "PowerModelsProtection", "PowerPlots", "PowerSimulations", "ProjectAssigner", "ProxSDP", "Pseudoseq", "QSimulator", "QXTools", "Qaintellect", "QuantumCliffordPlots", "QuantumControl", "QuantumPropagators", "QuantumStateBase", "QuasiMonteCarlo", "Queryverse", "RLEVectors", "RandomLinearAlgebraSolvers", "RatingCurves", "Recombinase", "Recommenders", "RegressionTables", "ReliabilityDiagrams", "RigorousCoupledWaveAnalysis", "Robotlib", "RvSpectMLPlots", "SASLib", "SMCExamples", "Santiago", "SegyIO", "SeisNoise", "Sherlock", "SimpleDrawing", "SphericalFunctions", "SpinGlassEngine", "StaticKernels", "StatsPlots", "Stipple", "StippleCharts", "StipplePlotly", "StippleUI", "StochasticIntegrals", "StomataModels", "Survival", "SwagUI", "SymbolicRegression", "TableDistances", "TheNumberLine", "TransferEntropy", "TransformUtils", "TuringCallbacks", "UnfoldMakie", "VCFTools", "VIDA", "VortexFilaments", "VueJS", "WaveletsExt", "Weave", "WiSER", "YAXArrayBase", "ZonalFlow"], vs = ":master")

@nanosoldier
Copy link
Copy Markdown
Collaborator

nanosoldier commented Jan 14, 2022

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

@timholy
Copy link
Copy Markdown
Member Author

timholy commented Jan 14, 2022

@nanosoldier runtests(["BasicInterpolators", "OptimKit"], vs = ":master")

@timholy
Copy link
Copy Markdown
Member Author

timholy commented Jan 14, 2022 •
edited
Loading

Locally this passes the remaining PkgEval issues, so I think this is good to go.

I have the next PR queued up behind this, so I'll leave this open for a week or so and then merge. As a preview, this + the next PR (which implements the changes to IR (de)compression) replaces #32705, except the increase in system image size is about 0.4% rather than 6%. So I think the concerns that doomed that approach don't apply.

@nanosoldier
Copy link
Copy Markdown
Collaborator

nanosoldier commented Jan 14, 2022

Your package evaluation job has completed - no new issues were detected. A full report can be found here.

@timholy timholy merged commit 4e83cce into master Jan 21, 2022
@timholy timholy deleted the teh/method_root_provenance branch January 21, 2022 09:24
@timholy timholy mentioned this pull request Jan 21, 2022
Merged
timholy added a commit that referenced this pull request Jan 21, 2022
@timholy
Fix asan for jl_precompile_toplevel_module
abbfcf0 
PR #43793 passed the buildkite test but the logs for #43881 show an
address sanitzer failure. Removing jl_precompile_toplevel_module from
jl_exported_data.inc fixes the error. For good measure, set it to NULL
at the point of definition, even though it gets nulled during
initialization.
@timholy timholy mentioned this pull request Jan 21, 2022
Merged
timholy added a commit that referenced this pull request Jan 21, 2022
@timholy
Fix asan for jl_precompile_toplevel_module (#43885)
a327428 
PR #43793 passed the buildkite test but the logs for #43881 show an
address sanitzer failure. Removing jl_precompile_toplevel_module from
jl_exported_data.inc fixes the error. For good measure, set it to NULL
at the point of definition, even though it gets nulled during
initialization.
This was referenced Jan 30, 2022
Merged
Closed
LilithHafner pushed a commit to LilithHafner/julia that referenced this pull request Feb 22, 2022
@timholy @LilithHafner
Track method roots by precompile module (JuliaLang#43793)
009bc4f 
For roots of a method added during incremental compilation, track the
toplevel module build_id. This will ultimately allow such roots to be
"relocatable" in compressed IR, pending future changes in compression.
LilithHafner pushed a commit to LilithHafner/julia that referenced this pull request Feb 22, 2022
@timholy @LilithHafner
Fix asan for jl_precompile_toplevel_module (JuliaLang#43885)
a7de47a 
PR JuliaLang#43793 passed the buildkite test but the logs for JuliaLang#43881 show an
address sanitzer failure. Removing jl_precompile_toplevel_module from
jl_exported_data.inc fixes the error. For good measure, set it to NULL
at the point of definition, even though it gets nulled during
initialization.
This was referenced Feb 25, 2022
Closed
Closed
LilithHafner pushed a commit to LilithHafner/julia that referenced this pull request Mar 8, 2022
@timholy @LilithHafner
Track method roots by precompile module (JuliaLang#43793)
2116f56 
For roots of a method added during incremental compilation, track the
toplevel module build_id. This will ultimately allow such roots to be
"relocatable" in compressed IR, pending future changes in compression.
LilithHafner pushed a commit to LilithHafner/julia that referenced this pull request Mar 8, 2022
@timholy @LilithHafner
Fix asan for jl_precompile_toplevel_module (JuliaLang#43885)
057ccdd 
PR JuliaLang#43793 passed the buildkite test but the logs for JuliaLang#43881 show an
address sanitzer failure. Removing jl_precompile_toplevel_module from
jl_exported_data.inc fixes the error. For good measure, set it to NULL
at the point of definition, even though it gets nulled during
initialization.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vtjnash vtjnash Awaiting requested review from vtjnash

@JeffBezanson JeffBezanson Awaiting requested review from JeffBezanson

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@timholy @nanosoldier