<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.9.5">Jekyll</generator><link href="https://jordinl.com/feed.xml" rel="self" type="application/atom+xml" /><link href="https://jordinl.com/" rel="alternate" type="text/html" /><updated>2024-06-02T13:49:04+00:00</updated><id>https://jordinl.com/feed.xml</id><title type="html">Jordi Noguera</title><subtitle></subtitle><entry><title type="html">How We Enhanced the User Experience for Mass Payouts with the PayPal API: A Comprehensive Case Study</title><link href="https://jordinl.com/posts/2024-03-09-how-we-improved-ux-for-mass-payouts-using-the-paypal-api-a-case-study" rel="alternate" type="text/html" title="How We Enhanced the User Experience for Mass Payouts with the PayPal API: A Comprehensive Case Study" /><published>2024-03-09T00:00:00+00:00</published><updated>2024-03-09T00:00:00+00:00</updated><id>https://jordinl.com/posts/how-we-improved-ux-for-mass-payouts-using-the-paypal-api-a-case-study</id><content type="html" xml:base="https://jordinl.com/posts/2024-03-09-how-we-improved-ux-for-mass-payouts-using-the-paypal-api-a-case-study"><![CDATA[<h3 id="introduction-addressing-a-critical-ux-challenge">Introduction: Addressing a Critical UX Challenge</h3>

<p>Last year, I identified an opportunity to improve the mass payout process via the PayPal API. This blog post is a case study detailing the challenges, solutions, and outcomes of this project.</p>

<h3 id="the-challenge-streamlining-the-mass-payout-process">The Challenge: Streamlining the Mass Payout Process</h3>

<p>Originally, our merchants experienced a cumbersome process for mass payouts. They would select payouts on a page-by-page basis and initiate them through the PayPal API. This method had several drawbacks:</p>

<ul>
  <li><strong>Limited Selection</strong>: Merchants could only process payments page by page due to pagination, making the task 
tedious for larger payout batches.</li>
  <li><strong>Lack of Feedback</strong>: After initiating a payout, merchants received a generic success message, with no immediate 
indication of the actual payout status. This lack of real-time feedback was particularly problematic if a payout failed, as merchants would only discover this by checking the failed payouts tab.</li>
</ul>

<video src="/assets/videos/paypal-ux-before.webm" width="100%" controls=""></video>

<h3 id="our-solution-enhancing-interactivity-and-feedback">Our Solution: Enhancing Interactivity and Feedback</h3>

<p>To address these issues, we embarked on a mission to overhaul the user experience. Our improvements focused on two key areas:</p>

<ul>
  <li><strong>Improved Selection Process</strong>: We introduced functionality to select all payouts across pages, complete with a summary 
of the total payouts and amounts. This change significantly streamlined the payout process for our users.</li>
  <li><strong>Real-Time Feedback</strong>: By leveraging Turbo Streams with Action Cable, we could now provide immediate feedback to the 
user about the status of their payout request. This meant users could see whether their payouts were successfully processed or if they failed due to issues such as insufficient funds.</li>
</ul>

<p>These enhancements were designed to make the mass payout process not only more efficient but also more transparent and user-friendly.</p>

<h3 id="technical-deep-dive-enhancing-the-mass-payout-process">Technical Deep Dive: Enhancing the Mass Payout Process</h3>

<h4 id="improved-selection-and-summary">Improved Selection and Summary</h4>

<p>To address pagination and streamline payout selections, we integrated Petite-Vue for its lightweight, reactive capabilities. Petite-Vue allowed us to dynamically update the summary panel as users select payouts, showing the total count and amount in real-time. This reactive solution improved the interface’s responsiveness, making the selection process across multiple pages seamless and user-friendly, all with minimal code complexity.</p>

<h4 id="real-time-feedback-with-turbo-streams-and-action-cable">Real-Time Feedback with Turbo Streams and Action Cable</h4>

<p>The integration of Turbo Streams with Action Cable was pivotal in providing real-time updates to users about the status of their payout requests. Here’s a brief overview:</p>

<ul>
  <li><strong>Backend Setup</strong>: When a payout submission is initiated, a background job is queued to process the request through the PayPal API. This background job is responsible for broadcasting the outcome (success or failure) to a unique Action Cable channel associated with the user session.</li>
  <li><strong>Frontend Listening</strong>: On the frontend, we subscribe to the Action Cable channel upon initiating the payout process. Turbo Streams listens for any broadcasts on this channel and dynamically updates the UI to reflect the status of the payout process, showing success or error messages as appropriate.</li>
  <li><strong>Error Handling</strong>: In cases where a payout fails (e.g., due to insufficient funds), the system captures the error from the PayPal API and broadcasts it to the user through the Action Cable channel. The UI then refreshes to display the relevant error message, allowing users to adjust their selections and try again.</li>
</ul>

<p>This approach not only streamlined the payout process but also significantly enhanced user engagement by keeping them informed throughout the process.</p>

<h3 id="the-outcome-a-more-intuitive-and-reliable-mass-payout-experience">The Outcome: A More Intuitive and Reliable Mass Payout Experience</h3>

<p>The implementation of these changes led to a noticeable improvement in the mass payout process:</p>

<ul>
  <li><strong>Increased Efficiency</strong>: The ability to select all payouts across pages reduced the time and effort required to 
initiate mass payouts.</li>
  <li><strong>Enhanced User Satisfaction</strong>: Immediate feedback on the status of payouts increased transparency and trust in the 
system.</li>
</ul>

<video src="/assets/videos/paypal-ux-after.webm" width="100%" controls=""></video>

<h3 id="lessons-learned-embracing-user-centric-design">Lessons Learned: Embracing User-Centric Design</h3>

<p>This project underscored the importance of listening to our users and being agile in our development process. By addressing a key pain point, we not only enhanced the functionality of our system but also significantly improved the user experience. This initiative serves as a reminder that in the digital world, continuous improvement is key to maintaining and enhancing user satisfaction and operational efficiency.</p>

<h3 id="conclusion-looking-forward">Conclusion: Looking Forward</h3>

<p>Refining the mass payout process using the PayPal API has been a significant learning experience for me, reinforcing the importance of user-centric solutions. As I continue to seek ways to improve and innovate, I value input from readers.</p>

<p>If you have any feedback on these changes, experiences with mass payouts, or ideas for further enhancements, I’d appreciate hearing from you. Please don’t hesitate to reach out through the contact form on my blog. Your insights can contribute to the ongoing conversation about best practices in the ever-changing world of technology.</p>]]></content><author><name></name></author><summary type="html"><![CDATA[A case study on enhancing the user experience for mass payouts using the PayPal API. Learn about the challenges, solutions, and outcomes of streamlining the payout process and providing real-time feedback to users.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://jordinl.com/assets/img/paypal-ux-improvements.png" /><media:content medium="image" url="https://jordinl.com/assets/img/paypal-ux-improvements.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">I once reduced response time by 80% with one line change</title><link href="https://jordinl.com/posts/2024-01-17-i-once-reduced-response-time-by-80-percent-with-one-line-change" rel="alternate" type="text/html" title="I once reduced response time by 80% with one line change" /><published>2024-01-17T00:00:00+00:00</published><updated>2024-01-17T00:00:00+00:00</updated><id>https://jordinl.com/posts/i-once-reduced-response-time-by-80-percent-with-one-line-change</id><content type="html" xml:base="https://jordinl.com/posts/2024-01-17-i-once-reduced-response-time-by-80-percent-with-one-line-change"><![CDATA[<p>Shortly after I started a job I noticed we were storing, for each of our customers, Stripe customer data as JSON in 
the DB. I actually said in a meeting that I thought it would be better to just store only the attributes we needed 
as columns instead, but I was told <em>not to worry about it and that it worked beautifully</em>. It was 
actually worse than that because Stripe allows to expand responses when querying their API so we were storing 
customer data with their subscriptions and payment method metadata; after that when we wanted to read this 
information we would hydrate Stripe 
objects from the <a href="https://github.com/stripe/stripe-ruby">Stripe ruby gem</a>.</p>

<p>So a few months go by and I’m asked to investigate some memory issues on production. I start chasing down the issue 
on my development machine, run some memory profiling and notice the Stripe gem is doing a lot of memory allocations. 
Thought that couldn’t be the issue so I start looking somewhere else. The actual pages with the biggest memory 
usage allowed embedding custom images and CSS so I thought the issue would be there. I spent 
some time looking into that, but I couldn’t find what was the cause. So I started commenting out one piece of code at 
a time until I noticed it was those Stripe JSON objects that were causing these huge memory spikes.</p>

<p>Everytime we wanted to check what type of Stripe subscription the customer had or if it was active we would hydrate 
a Stripe object with the subscription data from the DB. The temporary fix I did was to memoize the hydrated Stripe 
subscription so this inefficient process was done only once per request and not many times, I think I counted 6 
times in a single request.</p>

<p>On my development machine I noticed an 84% reduction in memory allocation for certain requests, so I knew it would 
improve 
things. When 
we pushed the code to production I couldn’t believe what I was seeing. I was looking at our APM tool to check 
performance and had to refresh the page a few times because I thought it wasn’t working right. Mean response time 
had dropped 80%, it also looked flat now instead of having lots of peaks and valleys.</p>

<p class="flex">
  <img src="/assets/img/memory-fix.png" alt="Memory Fix" />
  <small>
    The pink solid line represents new response time and the pink dotted line old response time
  </small>
</p>

<p class="flex">
  <img src="/assets/img/pat-bateman-approves.gif" alt="Pat Bateman approves" />
</p>]]></content><author><name></name></author><summary type="html"><![CDATA[Shortly after I started a job I noticed we were storing, for each of our customers, Stripe customer data as JSON in the DB. I actually said in a meeting that I thought it would be better to just store only the attributes we needed as columns instead, but I was told not to worry about it and that it worked beautifully. It was actually worse than that because Stripe allows to expand responses when querying their API so we were storing customer data with their subscriptions and payment method metadata; after that when we wanted to read this information we would hydrate Stripe objects from the Stripe ruby gem.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://jordinl.com/assets/img/memory-fix.png" /><media:content medium="image" url="https://jordinl.com/assets/img/memory-fix.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">How to generate random geocoordinates within a given radius using the Haversine Formula</title><link href="https://jordinl.com/posts/2019-02-15-how-to-generate-random-geocoordinates-within-given-radius" rel="alternate" type="text/html" title="How to generate random geocoordinates within a given radius using the Haversine Formula" /><published>2019-02-15T00:00:00+00:00</published><updated>2019-02-15T00:00:00+00:00</updated><id>https://jordinl.com/posts/how-to-generate-random-geocoordinates-within-given-radius</id><content type="html" xml:base="https://jordinl.com/posts/2019-02-15-how-to-generate-random-geocoordinates-within-given-radius"><![CDATA[<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-MML-AM_CHTML"></script>

<h3 id="introduction">Introduction</h3>

<p>On a previous project we needed to show markers on a map to indicate there was a property, but for privacy 
reasons we wanted to show the marker on the map close to the property but not at the exact location of the property. So
we needed to generate random points within a fixed distance of the original point location. As you would expect, I did 
some Googling first but, to my surprise, the solutions I found didn’t produce something that looked like a circle when 
generating many points at exactly a given radius, so they were far from good in my opinion.</p>

<p>In conclusion, I had to create a better solution and that’s when I decided to apply…</p>

<h3 id="the-haversine-formula">The Haversine Formula</h3>

<div>
  The Haversine Formula is used to calculate the spherical distance \(d\) on a sphere of radius \(R\) between two points 
  with latitude and longitude \(\left(\varphi_1, \lambda_1\right)\) and \(\left(\varphi_2, \lambda_2\right)\):
  
  $$hav \ \frac{d}{R} = hav \left(\varphi_2 - \varphi_1\right) + \cos \varphi_1 \cos \varphi_2 \ hav \left(\lambda_2 - \lambda_1\right)$$ 
  
  Where 
  
  $$hav \ \alpha = \frac{1 - \cos \alpha}{2}$$
  
  And therefore
  
  $$
    \begin{align}
    - \cos \frac{d}{R} &amp;= - \cos \left(\varphi_2 - \varphi_1\right) + \cos \varphi_1 \cos \varphi_2 \left(1 - \cos \left(\lambda_2 - \lambda_1 \right)\right) \\
    &amp;= - \sin \varphi_1 \sin \varphi_2 - \cos \varphi_1 \cos \varphi_2 \cos \left(\lambda_2 - \lambda_1 \right)
    \end{align}
  $$
  
  Finally
  
  $$
    d = R \arccos \left(\sin \varphi_1 \sin \varphi_2 + \cos \varphi_1 \cos \varphi_2 \cos \left(\lambda_2 - \lambda_1 \right)\right)
  $$
</div>

<h3 id="generating-random-points">Generating random points</h3>

<p>
  Instead of using the Haversine formula to find the distance \(d\) between two points \(\left(\varphi_1, \lambda_1\right)\) 
  and \(\left(\varphi_2, \lambda_2\right)\), we're going to use it to find all points \(\left(\varphi_2, \lambda_2\right)\)
  whose distance from \(\left(\varphi_1, \lambda_1\right)\) is \(d\).
</p>

<p>
  Note \(0 \lt d \lt \pi R\), \(-\frac{\pi}{2} \lt \varphi_i \le \frac{\pi}{2} \) and \(- \pi \lt \lambda_i \le \pi \). We're also
  going to define \(\Delta \varphi = \varphi_2 - \varphi_1\) and \(\Delta \lambda = \lambda_2 - \lambda_1 \) and will be 
  solving the equation for \(\Delta \varphi\) and \(\Delta \lambda\) instead. 
</p>

<p>
  So let's start with
  
  $$
    \cos \frac{d}{R} = \sin \varphi_1 \sin \varphi_2 + \cos \varphi_1 \cos \varphi_2 \cos \left(\lambda_2 - \lambda_1 \right)
  $$
  
  We can isolate \(\Delta \lambda\)
  
  $$
    \Delta \lambda = \pm \arccos \frac{\cos \frac{d}{R} - \sin \varphi_1 \sin \varphi_2}{\cos \varphi_1 \cos \varphi_2}
  $$
  
  And arccos only takes values between -1 and 1, therefore:
  
  $$
    -1 \le \frac{\cos \frac{d}{R} - \sin \varphi_1 \sin \varphi_2}{\cos \varphi_1 \cos \varphi_2} \le 1
  $$
  
  From the right hand side inequality
  
  $$
    \begin{align}
      \cos \frac {d}{R} \le \ &amp;\cos \varphi_1 \cos \varphi_2 + \sin \varphi_1 \sin \varphi_2 =\\
      &amp; \cos \left(\varphi_2 - \varphi_1\right) = \cos \Delta \varphi
    \end{align}
  $$
  
  Therefore, given \(0 \lt d \lt \pi R\)
  
  $$
    - \frac{d}{R} \le \Delta \varphi \le \frac {d}{R}
  $$
  
  From the left hand side inequality
  
  $$
    \begin{align}
      \cos \frac {d}{R} \ge \ &amp;- \cos \varphi_1 \cos \varphi_2 + \sin \varphi_1 \sin \varphi_2 =\\
      &amp; - \cos \left(\varphi_1 + \varphi_2\right) = - \cos \left(\Delta \varphi + 2 \varphi_1 \right)
    \end{align}
  $$
  
  Hence
  
  $$
    \cos \left(\Delta \varphi + 2 \varphi_1 \right) \ge - \cos \frac {d}{R} = \cos \left(\pi + \frac {d}{R}\right)
  $$
  
  Therefore, given \(0 \lt d \lt \pi R\) and \(-\frac{\pi}{2} \lt \varphi_i \le \frac{\pi}{2} \)
  
  $$
    - \frac{d}{R} \le \Delta \varphi \le \frac {d}{R}
  $$

  <b>In conclusion</b>, We can generate points \(\left(\varphi_2, \lambda_2\right)\) whose distance from 
  \(\left(\varphi_1, \lambda_1\right)\) is \(d\) with
  
  $$
    - \frac{d}{R} \le \Delta \varphi \le \frac {d}{R} \\
    \Delta \lambda = \pm \arccos \left(\frac{\cos \frac{d}{R} - \cos \Delta \varphi}{\cos \varphi_1 \cos \left(\Delta \varphi + \varphi_1\right)} + 1\right) \\
    \varphi_2 = \varphi_1 + \Delta \varphi, \ \lambda_2 = \lambda_1 + \Delta \lambda
  $$
  
  <b>Note 1</b> We won't get results for \(\Delta \lambda\) above when \(\varphi_1 = \pm \frac{\varphi}{2}\), that is
  for the poles. 
</p>

<p>
  <b>Note 2</b> We're using spherical coordinates, but the Earth is not a perfect sphere, so this just an approximation.
</p>

<h3 id="ruby-implementation">Ruby Implementation</h3>

<p>The code below will generate a new point that’s more than 100 meters away but less than 200 meters away from the 
original point. Notice we convert the input from degrees to radians and the output from radians to degrees. Also, it would
probably be a good idea to use the big decimal library to avoid floating point errors. And lastly, if this is for a web and 
you need map markers to be at the same spot across refreshes you will have to use a pseudo-random number generator.</p>

<p>
  <b>Edit (18-02-2019):</b> as pointed out in a 
  <a href="https://www.reddit.com/r/programming/comments/arx95x/how_to_generate_random_geocoordinates_within_a/">discussion</a> 
  on Reddit, to generate points uniformly across each circle, `\Delta \varphi` needs to be `d / R \ \cos x` where `x` is 
  a random number between `0` and `\pi`.
</p>

<p>
  <b>Edit (19-02-2019):</b> Also from the same discussion on Reddit, to generate a uniform ditribution accross the radius,
  we need to generate points at distance `\sqrt \left(x \left(d_max^2 - d_min^2\right) + d_min^2 \right)` where `x` is a random number 
  between 0 and 1.
</p>

<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">lat</span> <span class="o">=</span> <span class="p">(</span><span class="n">point</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">*</span> <span class="no">Math</span><span class="o">::</span><span class="no">PI</span> <span class="o">/</span> <span class="mi">180</span><span class="p">).</span><span class="nf">to_d</span>
<span class="n">lon</span> <span class="o">=</span> <span class="p">(</span><span class="n">point</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">*</span> <span class="no">Math</span><span class="o">::</span><span class="no">PI</span> <span class="o">/</span> <span class="mi">180</span><span class="p">).</span><span class="nf">to_d</span>
<span class="n">max_distance</span> <span class="o">=</span> <span class="mi">200</span><span class="p">.</span><span class="nf">to_d</span>
<span class="n">min_distance</span> <span class="o">=</span> <span class="mi">100</span><span class="p">.</span><span class="nf">to_d</span>
<span class="n">earth_radius</span> <span class="o">=</span> <span class="mi">6_371_000</span><span class="p">.</span><span class="nf">to_d</span>
<span class="n">distance</span> <span class="o">=</span> <span class="no">Math</span><span class="p">.</span><span class="nf">sqrt</span><span class="p">(</span><span class="nb">rand</span> <span class="o">*</span> <span class="p">(</span><span class="n">max_distance</span> <span class="o">**</span> <span class="mi">2</span> <span class="o">-</span> <span class="n">min_distance</span> <span class="o">**</span> <span class="mi">2</span><span class="p">)</span> <span class="o">+</span> <span class="n">min_distance</span> <span class="o">**</span> <span class="mi">2</span><span class="p">)</span>

<span class="n">delta_lat</span> <span class="o">=</span> <span class="no">Math</span><span class="p">.</span><span class="nf">cos</span><span class="p">(</span><span class="nb">rand</span> <span class="o">*</span>  <span class="no">Math</span><span class="o">::</span><span class="no">PI</span><span class="p">)</span> <span class="o">*</span> <span class="n">distance</span> <span class="o">/</span> <span class="n">earth_radius</span>
<span class="n">sign</span> <span class="o">=</span> <span class="nb">rand</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="o">*</span> <span class="mi">2</span> <span class="o">-</span> <span class="mi">1</span>
<span class="n">delta_lon</span> <span class="o">=</span> <span class="n">sign</span> <span class="o">*</span> <span class="no">Math</span><span class="p">.</span><span class="nf">acos</span><span class="p">(</span>
  <span class="p">((</span><span class="no">Math</span><span class="p">.</span><span class="nf">cos</span><span class="p">(</span><span class="n">distance</span><span class="o">/</span><span class="n">earth_radius</span><span class="p">)</span> <span class="o">-</span> <span class="no">Math</span><span class="p">.</span><span class="nf">cos</span><span class="p">(</span><span class="n">delta_lat</span><span class="p">))</span> <span class="o">/</span>
    <span class="p">(</span><span class="no">Math</span><span class="p">.</span><span class="nf">cos</span><span class="p">(</span><span class="n">lat</span><span class="p">)</span> <span class="o">*</span> <span class="no">Math</span><span class="p">.</span><span class="nf">cos</span><span class="p">(</span><span class="n">delta_lat</span> <span class="o">+</span> <span class="n">lat</span><span class="p">)))</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span>

<span class="p">[(</span><span class="n">lat</span> <span class="o">+</span> <span class="n">delta_lat</span><span class="p">)</span> <span class="o">*</span> <span class="mi">180</span> <span class="o">/</span> <span class="no">Math</span><span class="o">::</span><span class="no">PI</span><span class="p">,</span> <span class="p">(</span><span class="n">lon</span> <span class="o">+</span> <span class="n">delta_lon</span><span class="p">)</span> <span class="o">*</span> <span class="mi">180</span> <span class="o">/</span> <span class="no">Math</span><span class="o">::</span><span class="no">PI</span><span class="p">]</span>
</code></pre></div></div>

<h3 id="examples">Examples</h3>

<p class="center">
  <img src="/assets/img/eiffel-tower.png" alt="Eiffel Tower" />
  Points between 100 meters and 200 meters away from the Eiffel Tower, Paris (France)
</p>

<p class="center">
  <img src="/assets/img/quito.png" alt="Quito" />
  Points between 1.5km and 2km away from Quito (Ecuador)
</p>

<p class="center">
  <img src="/assets/img/greenwich.png" alt="Greenwich, London" />
  Points between 8km and 10km away from the Greenwich Observatory, London (UK)
</p>

<p class="center">
  <img src="/assets/img/buenos-aires.png" alt="Buenos Aires, Argentina" />
  Points between 50km and 100km away from Buenos Aires (Argentina)
</p>

<p class="center">
  <img src="/assets/img/denver.png" alt="Denver, Colorado" />
  Points between 500km and 1000km away from Denver (Colorado)
</p>]]></content><author><name></name></author><summary type="html"><![CDATA[]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://jordinl.com/assets/img/eiffel-tower.png" /><media:content medium="image" url="https://jordinl.com/assets/img/eiffel-tower.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">How to run periodic jobs with Sidekiq for free</title><link href="https://jordinl.com/posts/2018-08-06-how-to-run-periodic-jobs-with-sideqik-for-free" rel="alternate" type="text/html" title="How to run periodic jobs with Sidekiq for free" /><published>2018-08-06T00:00:00+00:00</published><updated>2018-08-06T00:00:00+00:00</updated><id>https://jordinl.com/posts/how-to-run-periodic-jobs-with-sideqik-for-free</id><content type="html" xml:base="https://jordinl.com/posts/2018-08-06-how-to-run-periodic-jobs-with-sideqik-for-free"><![CDATA[<p>One of the features not included in the free version of <strong>Sidekiq</strong> is running periodic jobs (or cron jobs or recurring jobs). There are gems that offer this functionality as an add-on, but it turns out that it can be implemented in less that 40 lines of code, so there is no need to pull an additional bloated dependency.</p>

<p>All we need is a <em><strong>Scheduler</strong></em> worker that when you run it:</p>

<ul>
  <li>It knows when to run other workers.</li>
  <li>It re-schedules itself to be executed after some time (1 minute, 10 minutes, 1hour, … whatever is necessary).</li>
</ul>

<p>Also, we need to kick off the <strong><em>Scheduler</em></strong> worker for the first time in the <strong>Sidekiq</strong> initializer.</p>

<p>Below you can find the implementation for the <strong><em>SchedulerWorker</em></strong> class and the changes to the <strong>Sidekiq</strong> initializer. With the current implementation of the <strong><em>SchedulerWorker</em></strong> it will run every minute and check if there are workers that must be run or not. The workers are configured in the <strong><em>SCHEDULE</em></strong> hash, containing the periodic workers as keys and the values are lambdas that return true when the worker must be run. For instance, in the code below, <strong><em>FirstWorker</em></strong> will run 5 minutes after the hour every hour. <strong><em>SecondWorker</em></strong> will run every day at 9AM. <strong><em>ThirdWorker</em></strong> will run every 10 minutes.</p>

<p>Enjoy.</p>

<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/workers/scheduler_worker.rb</span>

<span class="k">class</span> <span class="nc">SchedulerWorker</span>
  <span class="kp">include</span> <span class="no">Sidekiq</span><span class="o">::</span><span class="no">Worker</span>
  <span class="n">sidekiq_options</span> <span class="ss">queue: </span><span class="s1">'critical'</span>

  <span class="no">SCHEDULE</span> <span class="o">=</span> <span class="p">{</span>
    <span class="no">FirstWorker</span>  <span class="o">=&gt;</span> <span class="o">-&gt;</span> <span class="p">(</span><span class="n">time</span><span class="p">)</span> <span class="p">{</span> <span class="n">time</span><span class="p">.</span><span class="nf">min</span> <span class="o">==</span> <span class="mi">5</span> <span class="p">},</span>
    <span class="no">SecondWorker</span> <span class="o">=&gt;</span> <span class="o">-&gt;</span> <span class="p">(</span><span class="n">time</span><span class="p">)</span> <span class="p">{</span> <span class="n">time</span><span class="p">.</span><span class="nf">min</span> <span class="o">==</span> <span class="mi">0</span> <span class="o">&amp;&amp;</span> <span class="n">time</span><span class="p">.</span><span class="nf">hour</span> <span class="o">==</span> <span class="mi">9</span> <span class="p">},</span>
    <span class="no">ThirdWorker</span>  <span class="o">=&gt;</span> <span class="o">-&gt;</span> <span class="p">(</span><span class="n">time</span><span class="p">)</span> <span class="p">{</span> <span class="n">time</span><span class="p">.</span><span class="nf">min</span> <span class="o">%</span> <span class="mi">10</span> <span class="o">==</span> <span class="mi">0</span> <span class="p">},</span>
  <span class="p">}</span>

  <span class="k">def</span> <span class="nf">perform</span>
    <span class="n">execution_time</span> <span class="o">=</span> <span class="no">Time</span><span class="p">.</span><span class="nf">zone</span><span class="p">.</span><span class="nf">now</span>
    <span class="n">execution_time</span> <span class="o">-=</span> <span class="n">execution_time</span><span class="p">.</span><span class="nf">sec</span>

    <span class="nb">self</span><span class="p">.</span><span class="nf">class</span><span class="p">.</span><span class="nf">perform_at</span><span class="p">(</span><span class="n">execution_time</span> <span class="o">+</span> <span class="mi">60</span><span class="p">)</span> <span class="k">unless</span> <span class="n">scheduled?</span>

    <span class="no">SCHEDULE</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="p">(</span><span class="n">worker_class</span><span class="p">,</span> <span class="n">schedule_lambda</span><span class="p">)</span><span class="o">|</span>
      <span class="n">worker_class</span><span class="p">.</span><span class="nf">perform_async</span> <span class="k">if</span> <span class="o">!</span><span class="n">scheduled?</span><span class="p">(</span><span class="n">worker_class</span><span class="p">)</span> <span class="o">&amp;&amp;</span> <span class="n">schedule_lambda</span><span class="p">.</span><span class="nf">call</span><span class="p">(</span><span class="n">execution_time</span><span class="p">)</span>
    <span class="k">end</span>
  <span class="k">end</span>

  <span class="k">def</span> <span class="nf">scheduled?</span><span class="p">(</span><span class="n">worker_class</span> <span class="o">=</span> <span class="nb">self</span><span class="p">.</span><span class="nf">class</span><span class="p">)</span>
    <span class="n">scheduled_workers</span><span class="p">[</span><span class="n">worker_class</span><span class="p">.</span><span class="nf">name</span><span class="p">]</span>
  <span class="k">end</span>

  <span class="kp">private</span>

  <span class="k">def</span> <span class="nf">scheduled_workers</span>
    <span class="vi">@scheduled_workers</span> <span class="o">||=</span> <span class="no">Sidekiq</span><span class="o">::</span><span class="no">ScheduledSet</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">entries</span><span class="p">.</span><span class="nf">each_with_object</span><span class="p">({})</span> <span class="k">do</span> <span class="o">|</span><span class="n">item</span><span class="p">,</span> <span class="nb">hash</span><span class="o">|</span>
      <span class="nb">hash</span><span class="p">[</span><span class="n">item</span><span class="p">[</span><span class="s1">'class'</span><span class="p">]]</span> <span class="o">=</span> <span class="kp">true</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># config/initializers/sidekiq.rb</span>

<span class="no">Sidekiq</span><span class="p">.</span><span class="nf">configure_server</span> <span class="k">do</span> <span class="o">|</span><span class="n">config</span><span class="o">|</span>
  <span class="n">config</span><span class="p">.</span><span class="nf">on</span><span class="p">(</span><span class="ss">:startup</span><span class="p">)</span> <span class="k">do</span>
    <span class="no">SchedulerWorker</span><span class="p">.</span><span class="nf">perform_async</span> <span class="k">unless</span> <span class="no">SchedulerWorker</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">scheduled?</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>]]></content><author><name></name></author><summary type="html"><![CDATA[One of the features not included in the free version of Sidekiq is running periodic jobs (or cron jobs or recurring jobs). There are gems that offer this functionality as an add-on, but it turns out that it can be implemented in less that 40 lines of code, so there is no need to pull an additional bloated dependency.]]></summary></entry><entry><title type="html">How to calculate business time between two times efficiently</title><link href="https://jordinl.com/posts/2017-03-05-how-to-calculalate-business-time-between-two-times-efficiently" rel="alternate" type="text/html" title="How to calculate business time between two times efficiently" /><published>2017-03-05T00:00:00+00:00</published><updated>2017-03-05T00:00:00+00:00</updated><id>https://jordinl.com/posts/how-to-calculalate-business-time-between-two-times-efficiently</id><content type="html" xml:base="https://jordinl.com/posts/2017-03-05-how-to-calculalate-business-time-between-two-times-efficiently"><![CDATA[<p>Given a company’s work schedule and two arbitrary times in two arbitrary days, we want to calculate the amount of time between these two times that falls inside the company’s work schedule (accounting for holidays too).</p>

<h3 id="example">Example</h3>

<p>Given a company based in the United States that opens Monday-Friday 9:00–17:00, how much time is there between Monday November 21st at 14:00 and Thursday December 1st at 11:00? Notice that Thursday November 24th is Thanksgiving and the company is not open:</p>

<p><img src="/assets/img/business-time/calendar.png" alt="Calendar" /></p>

<p>So, in this case, we have:</p>

<ul>
  <li>Nov 21st: 3 hours from 14:00 to 17:00.</li>
  <li>6 full working days: 6 * 8 hours = 48 hours.</li>
  <li>Thursday Dec 1st: 2 hours from 9:00 to 11:00.</li>
</ul>

<p>Total: 53 hours.</p>

<p><img src="/assets/img/business-time/hours.png" alt="Hours" /></p>

<h3 id="brute-force-solution">Brute force solution</h3>

<p>Iterate through all the dates in the interval:</p>

<ul>
  <li>If the date is not a weekday count 0.</li>
  <li>If the date is a holiday count 0.</li>
  <li>If the date is a weekday and not a holiday calculate business hours in day. For dates days outside of the edges of the interval it’s just closing time-opening time. For dates on the edges we would need to calculate the intersection.</li>
</ul>

<p><strong>Complexity</strong>: As you can see we need to loop through all dates inside of the interval and for each of the dates we need to check if it’s a holiday. Therefore the complexity is NxM, where N is the distance between the dates and M is the length of the array that contains the holidays. We could use a Hash to store the holidays and this would take down the complexity to linear.</p>

<h3 id="optimal-solution">Optimal solution</h3>

<h4 id="step-1">Step 1</h4>

<p>Calculate business time inside the interval excluding the edges, this way we can count full working days and weeks.</p>

<p>Let’s call <em>W</em> the set of <em>weekdays</em> between two dates. And let’s call <em>H</em> the set of <em>holidays</em> between two dates. The number of <em>workdays</em> between these two dates will be <em>W-(W∩H)</em>.</p>

<p><img src="/assets/img/business-time/intersection.png" alt="Intersection" /></p>

<p><strong>We now the <em>schedule</em> ahead of time, therefore we can calculate the following before hand:</strong></p>

<ul>
  <li>How much business time there is during a natural week. (eg 40 hours)</li>
  <li>How much time there is between any combination of two weekdays, for instance, between Monday and Tuesday there are 16 hours, Monday and Wednesday there are 24 hours, Sunday and Wednesday there are 24 hours, …</li>
</ul>

<p>This way we can calculate the total time during workdays as <em>(number of weeks x hours per week) + (hours between weekday of first day and weekday of last day)</em>. And therefore we can calculate it in constant time.</p>

<p><strong>Calculate amount of time during holidays on weekdays between two dates:</strong></p>

<p>We will also know the <em>holidays</em> ahead of time and we will do some calculations ahead of time too. We will iterate through every date between <em>first holiday</em> and <em>last holiday</em> and we will calculate the following:</p>

<ul>
  <li>Whether that date is a holiday.</li>
  <li>Number of <em>“business hours”</em> that occurred on <em>holidays</em> before that date. These are not actual business hours, but they would’ve been if they hadn’t been on a holiday.</li>
</ul>

<p>So the pseudo-code for this calculation would look like this:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>holidays = [...] // we are passed an array with all the holidays
holiday_hours = 0
holiday_info = {}for date in range(holidays.first, holidays.last) do
  holiday = date.inside?(holidays)
  holiday_info[date] = {
    holiday: holiday,
    holiday_hours_before: holiday_hours
  }  if holiday and date.weekday? do
    holiday_hours = holiday_hours + business_hours_in(date)  
  end
end  
</code></pre></div></div>

<p>This way if we want to calculate how many <em>“business hours”</em> fell during <em>holidays</em> between <em>date1</em> and <em>date2</em>, we will just need to calculate:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>holiday_info[date2 + 1] - holiday_info[date1]
</code></pre></div></div>

<p>Which can be calculated in constant time.</p>

<h4 id="step-2">Step 2</h4>

<p>Calculate business time on the edges of the interval:</p>

<ul>
  <li>If the edge doesn’t fall on a weekday count 0.</li>
  <li>If the edge is a holiday count 0.</li>
  <li>Otherwise, for left edge calculate hours between <em>left edge start time</em> and <em>end of business day</em>. For <em>right edge</em> calculate hours between <em>start of business day</em> and <em>right edge end time</em>.</li>
</ul>

<p>All this can be calculated in constant time too.</p>

<h3 id="operating-hours-gem">Operating Hours gem</h3>

<h2 id="open-source">Open Source</h2>

<h3 id="operating-hours">Operating Hours</h3>

<p><strong>Update 2020-11-06:</strong> The organization that hosted the <strong>operating_hours</strong> gem was bought by another company and unfortunately all its repos, including <strong>operating_hours</strong> were deleted.</p>

<p><a href="https://github.com/spreemo/operating_hours">Operating Hours</a> is a Ruby Gem that makes time calculations based on business hours. It’s inspired by <a href="https://github.com/bokmann/business_time">business_time</a> but with improved performance. See tables below for a comparison of the performance of calculating the business time between two random dates:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Business time:
Distance        | Execution time  | Result
--------------- | --------------- | ---------------
1 year apart    | 24.077092       | 7257600.0
6 months apart  | 12.693819       | 3744000.0
3 months apart  | 6.426698        | 1814400.0
1 month apart   | 2.466015        | 576000.0
2 weeks apart   | 1.459892        | 259200.0
1 week apart    | 1.00986         | 115200.0
2 days apart    | 0.776951        | 28800.0
1 day apart     | 0.839757        | 28800.0
Same day        | 0.595277        | 14400.0

Operating Hours:
Distance        | Execution time  | Result
--------------- | --------------- | ---------------
1 year apart    | 0.00949         | 7257600
6 months apart  | 0.01707         | 3744000
3 months apart  | 0.012577        | 1814400
1 month apart   | 0.029238        | 576000
2 weeks apart   | 0.018997        | 259200
1 week apart    | 0.019849        | 115200
2 days apart    | 0.010557        | 28800
1 day apart     | 0.016287        | 28800
Same day        | 0.020898        | 14400
</code></pre></div></div>]]></content><author><name></name></author><summary type="html"><![CDATA[Given a company’s work schedule and two arbitrary times in two arbitrary days, we want to calculate the amount of time between these two times that falls inside the company’s work schedule (accounting for holidays too).]]></summary></entry></feed>