Fixed off by one and other issues (affecting models, log-likelihood computation and predictions).#48
Merged
iLampard merged 11 commits intoant-research:mainfrom Oct 14, 2024
Merged
Conversation
Three changes being made to log-likelihood computation: 1. Changed the computation of logL only using type_mask to be more memory-efficient and can be extended to higher-dimensional mark space. This affects all models. 2. Use trapezoid rule if the integral is estimated using grid points for each inter-event interval, because the last grid point on each interval always overlaps with the left intensity at event time 3. Add the option to use Monte Carlo samples in estimating the integral so the estimates are unbiased. The list of `batch` and corresponding masks are updated. Co-Authored-By: ajboyd2 <ajboyd2@calpoly.edu>
1. Fixed the off by one shifting issue. 2. Removed `max_seq_length` in `forward`. 3. Integrated `compute_states_at_sample_times` that was intended to compute intensity into `evolve_and_get_intentsity`. Co-Authored-By: ajboyd2 <ajboyd2@calpoly.edu>
1. Fixed the off by one issue. 2. Removed `max_decay_time` and `num_steps` in `forward`. 3. Add mark-specific learnable beta parameters for Softplus to be consistent with the paper. Co-Authored-By: ajboyd2 <ajboyd2@calpoly.edu>
1. Fixed off by one issue (with dts that causes leakage). 2. Added mark-specific beta for Softplus. 3. Match attention mask to `compute_intensities_at_sample_times` and the updated `event_tokenizer`. Co-Authored-By: ajboyd2 <11725441+ajboyd2@users.noreply.github.com>
1. Added mark-specific beta for Softplus. 2. Used mark sequences to compute log-likelihood and match attention mask to `compute_intensities_at_sample_times` and the updated `event_tokenizer`. Co-Authored-By: ajboyd2 <11725441+ajboyd2@users.noreply.github.com>
1. Added mark-specific beta for Softplus (match the paper). 2. Fixed compatibility for GPU. 3. Used mark sequences to compute log-likelihood and match attention mask to `compute_intensities_at_sample_times` and the updated `event_tokenizer`. Co-Authored-By: ajboyd2 <11725441+ajboyd2@users.noreply.github.com>
1. Fixed the shifting issue in dts. 2. Used mark sequences to compute log-likelihood. Co-Authored-By: ajboyd2 <11725441+ajboyd2@users.noreply.github.com>
1. Fixed the off by one shifting issue. 2. Used mark sequences to compute log-likelihood. 3. Now allows for having marked intensities. Co-Authored-By: ajboyd2 <11725441+ajboyd2@users.noreply.github.com>
1. Fixed three sources of data leakage: (1) For the parameterized distribution, the running statistics of dts were computed using the current batch's dts which effectively cheats by knowing ahead of time the scale of the targeted dts. (2) The shifting of dts misaligned with the marks. (3) The log-likelihood of marks is computed against the wrong marks due to incorrect shifting - specifically the marks that they conditioned on. 2. Corrected the log likelihood that was reported on an incorrect scale. It originally was ll.sum(dim=-1).mean() when it should just be ll.sum(). 3. Now supports next event prediction based on the parameterized distribution. Co-Authored-By: ajboyd2 <11725441+ajboyd2@users.noreply.github.com>
Co-Authored-By: ajboyd2 <11725441+ajboyd2@users.noreply.github.com>
1. Fixed the shift by one issue (especially with dts) that matches other model implementations. 2. With multiple accepted sampled time, the events were originally sampled conditioned on the expected time \lambda(k | t_avg), while they should be sampled conditioned on *each* accepted sampled time \lambda(k | t_j) at all t_j. Then we normalize the intensity values to Categorical distributions, aggregate the distributions, and then take argmax for the final prediction of mark. This is now fixed with explanation in comments. Co-Authored-By: ajboyd2 <11725441+ajboyd2@users.noreply.github.com>
Collaborator
|
Thanks. As discussed, we will issue a tech report soon to explain the new version of easytpp. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
event_llto be more memory efficient, so it can be extended to higher dimensional mark space.All these changes are made collaboratively with @ajboyd2.