BI-2043 - Improve Observation Overwrite Performance#329
Conversation
nickpalladino
left a comment
There was a problem hiding this comment.
I was able to update using a file with a single row but when I tried another update with a bunch of rows it was hanging on loading prior to generating the preview table.
shawn_exp_4.xls
shawn_exp_4_update.xls
shawn_exp_4_update_2.xls
|
I was looking through some redis stuff and looked like we aren't caching observation units or observations. What do you think about caching those and the impacts on overwrite performance? Maybe a new card? |
@nickpalladino I created a v0.10 card to cache Observations and ObservationUnits.
@nickpalladino I uploaded those files in order, the last file took about 30 seconds to get to preview. Did it take longer than that for you? |
Description
Story: BI-2043
ExperimentProcessor::initNewBrapiDatawas callingfetchOrCreateObservationPIOin a nested loop for each phenotype column for each row, which in turn was callingfetchExistingObservations, an expensive method which resulted in 2 network calls to the backing BrAPI service. I was able to pull thefetchExistingObservationscall outside the loops without changing the behavior of the code (except the BrAPI calls are now made just once). Performance is greatly improved.Dependencies
bi-web: release/0.9
bi-api: bug/BI-2043
Testing
Checklist: