{"id":4515,"date":"2025-12-04T20:45:21","date_gmt":"2025-12-04T15:15:21","guid":{"rendered":"https:\/\/blog.spike.sh\/?p=4515"},"modified":"2026-01-09T13:16:39","modified_gmt":"2026-01-09T07:46:39","slug":"postmortem-on-call-system-discrepancy","status":"publish","type":"post","link":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/","title":{"rendered":"Postmortem of On-Call System Discrepancy"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Summary<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">On December 4th, 2025, we discovered an issue where the person shown as on-call on the dashboard didn&#8217;t match who was scheduled in the calendar.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When we started investigating, we learnt that&nbsp; this only affected schedules with weekly rotations or weekly layers combined with custom timings.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The root cause was wild: our old code had a bug that ignored schedule start dates for weekly layers, activating them before the start time instead of waiting for the configured start time. When users created schedules months ago, the calendar UI showed what they expected, so nobody noticed anything wrong. Then we fixed the bug in a recent release, the calendar corrected itself to show the right sequence, but our queuing system was still running jobs based on the old, incorrect sequence. The result was pure chaos. Calendar showed one person on-call, but another person was actually on-call who was also receiving alerts.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Detection<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A user reported this issue earlier today- the person on the platform didn&#8217;t match their calendar, and alerts were going to the wrong user. We thought it might be a queuing error from a recent deployment, so we reset their schedule. It worked. Problem solved, right?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong><em>Nope<\/em><\/strong>. Another customer reported the same issue. That&#8217;s when we knew this was systemic. We declared an incident and started investigating.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Debug<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">We tried reproducing it in our test environment but everything worked fine. Then we had an idea &#8211; what if we replicated one of the actual problematic schedules exactly? So we did.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The calendar rendered correctly, but something was off. Users were going on-call back-to-back instead of alternating, for example ,&nbsp; Person A would be on-call, then Person A again, then Person B, then Person B, then Person C.&nbsp; Not wrong, but&#8230; weird<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"765\" data-attachment-id=\"4519\" data-permalink=\"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/image2-2\/\" data-orig-file=\"https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/image2-1.png\" data-orig-size=\"1970,1472\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"image2\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/image2-1-1024x765.png\" src=\"https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/image2-1-1024x765.png\" alt=\"Incorrect sequence on on-call schedule\" class=\"wp-image-4519\" srcset=\"https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/image2-1-1024x765.png 1024w, https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/image2-1-300x224.png 300w, https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/image2-1-768x574.png 768w, https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/image2-1-1536x1148.png 1536w, https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/image2-1-400x300.png?crop=1 400w, https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/image2-1-200x150.png?crop=1 200w, https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/image2-1-1200x897.png 1200w, https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/image2-1.png 1970w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">We dug into the history and found these schedules had been running since June, 2025. We plotted what the calendar should look like according to our current code &#8211; it matched perfectly. So why were customers complaining?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here&#8217;s where we got creative: we pulled up an old release from when these schedules were created and rendered the same schedule using that old code. Suddenly everything made sense.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The old code showed a completely different sequence. What looked weird in our current code looked totally normal in the old code.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong><em>The bug:<\/em><\/strong> Our legacy code ignored start dates when activating weekly interval layers. Set a weekly rotation to start June 20? The old code started it a few days early &#8211; on June 16. The actual on-call shifts began earlier, which resulted in starting one layer earlier and threw off the entire on-call sequence.&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Schedules with weekly interval layers<\/li>\n\n\n\n<li>Schedules with weekly layers combined with custom timings<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Daily rotations and other interval types weren&#8217;t impacted.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The fix that broke things: We recently fixed this bug so weekly layers now properly respect start dates. The calendar immediately corrected itself. But our queuing system doesn&#8217;t pre-calculate all shifts &#8211; it waits for one shift to end, then calculates the next person. So when we deployed the fix:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The calendar showed the corrected rotation (Person A on-call)<\/li>\n\n\n\n<li>The queue was still running a shift from the old rotation (Person B receiving alerts)<\/li>\n\n\n\n<li>They wouldn&#8217;t sync until the current shift ended<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">That&#8217;s why resetting the schedule worked temporarily &#8211; it cleared the old queued jobs.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Recovery<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Once we understood the issue, the fix was straightforward:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Identified all affected schedules (weekly interval layers or weekly + custom timings created between when the bug was introduced and when we fixed it)<\/li>\n\n\n\n<li>Synced them all with the correct data<\/li>\n\n\n\n<li>Cleared pending queue jobs operating on the old sequence<\/li>\n\n\n\n<li>Verified the calendar matched who was actually on-call<\/li>\n\n\n\n<li>Resolved the status page incident<\/li>\n\n\n\n<li>Prepared to notify affected customers<\/li>\n<\/ol>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Lessons Learned<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The big takeaway: Bugs can hide for months when the UI masks the underlying issue, especially when they only affect specific configurations like weekly layers. Then when you fix them, the fix itself can expose the problem in ways that impact customers.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What we need to do better:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Better testing:<\/strong> End-to-end tests that validate both calendar display AND alert routing across different layer types (daily, weekly, custom). Test alignment between calendar and queue states for each interval type.<\/li>\n\n\n\n<li><strong>Rethinking the on-call engine:<\/strong> A new on-call engine is in the cards. One where we can properly handle cross-timezone settings, store calendar data for both the past and near future, and compute the rest in real-time. Users love the infinite on-call calendar and we&#8217;re not complaining about it either &#8211; we&#8217;ve seen teams add on-call overrides for Christmas in April, and that seamlessness is worth keeping.&nbsp;<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"wp-block-paragraph\">If you have feedback or thoughts, please reach out to us at <a href=\"mailto:hello@spike.sh\">hello@spike.sh<\/a>. I&#8217;m also available on <a href=\"https:\/\/www.linkedin.com\/in\/its-damanpreet\/\">LinkedIn<\/a> or <a href=\"https:\/\/twitter.com\/your-handle\"><\/a><a href=\"https:\/\/x.com\/@its_damanpreet\">@its_damanpreet<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>On December 4th, 2025, we identified a critical discrepancy between displayed on-call schedules and actual alert routing for weekly rotations. This postmortem details how a recent bug fix exposed months of underlying system misalignment, our investigation process, and key lessons.<\/p>\n","protected":false},"author":263547071,"featured_media":4520,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_crdt_document":"","_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","_lmt_disableupdate":"","_lmt_disable":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":true,"token":"eyJpbWciOiJodHRwczpcL1wvYmxvZy5zcGlrZS5zaFwvd3AtY29udGVudFwvdXBsb2Fkc1wvMjAyNVwvMTJcL1Bvc3Rtb3JldGVtLTEwMjR4NTU1LnBuZyIsInR4dCI6IlBvc3Rtb3J0ZW0gb2YgT24tQ2FsbCBTeXN0ZW0gRGlzY3JlcGFuY3kiLCJ0ZW1wbGF0ZSI6ImhpZ2h3YXkiLCJmb250IjoiIiwiYmxvZ19pZCI6MjMzMTM4OTAwfQ.peFZxD3qAISX_B4rA1CMOcU4DPDor5kRRWDhB8XA3qwMQ"},"version":2},"_wpas_customize_per_network":false,"jetpack_post_was_ever_published":false},"categories":[1444],"tags":[],"class_list":["post-4515","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-postmortem"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Postmortem of On-Call System Discrepancy<\/title>\n<meta name=\"description\" content=\"Read the postmortem on on-call system discrepancy.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Postmortem of On-Call System Discrepancy\" \/>\n<meta property=\"og:description\" content=\"Read the postmortem on on-call system discrepancy.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/\" \/>\n<meta property=\"og:site_name\" content=\"Spike&#039;s blog\" \/>\n<meta property=\"article:published_time\" content=\"2025-12-04T15:15:21+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-01-09T07:46:39+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/Postmoretem.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1040\" \/>\n\t<meta property=\"og:image:height\" content=\"564\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Damanpreet\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Damanpreet\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/\"},\"author\":{\"name\":\"Damanpreet\",\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/#\\\/schema\\\/person\\\/4bc971837058eacf02e9ef12fb625155\"},\"headline\":\"Postmortem of On-Call System Discrepancy\",\"datePublished\":\"2025-12-04T15:15:21+00:00\",\"dateModified\":\"2026-01-09T07:46:39+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/\"},\"wordCount\":806,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/blog.spike.sh\\\/wp-content\\\/uploads\\\/2025\\\/12\\\/Postmoretem.png\",\"articleSection\":[\"Postmortem\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/\",\"url\":\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/\",\"name\":\"Postmortem of On-Call System Discrepancy\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/blog.spike.sh\\\/wp-content\\\/uploads\\\/2025\\\/12\\\/Postmoretem.png\",\"datePublished\":\"2025-12-04T15:15:21+00:00\",\"dateModified\":\"2026-01-09T07:46:39+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/#\\\/schema\\\/person\\\/4bc971837058eacf02e9ef12fb625155\"},\"description\":\"Read the postmortem on on-call system discrepancy.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/#primaryimage\",\"url\":\"https:\\\/\\\/blog.spike.sh\\\/wp-content\\\/uploads\\\/2025\\\/12\\\/Postmoretem.png\",\"contentUrl\":\"https:\\\/\\\/blog.spike.sh\\\/wp-content\\\/uploads\\\/2025\\\/12\\\/Postmoretem.png\",\"width\":1040,\"height\":564,\"caption\":\"blog cover postmortem of on-call system discrepancy\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/postmortem-on-call-system-discrepancy\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/blog.spike.sh\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Postmortem of On-Call System Discrepancy\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/#website\",\"url\":\"https:\\\/\\\/blog.spike.sh\\\/\",\"name\":\"Spike&#039;s blog\",\"description\":\"Learnings and opinions in a changing world\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/blog.spike.sh\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/blog.spike.sh\\\/#\\\/schema\\\/person\\\/4bc971837058eacf02e9ef12fb625155\",\"name\":\"Damanpreet\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/9a938d08ab9251a99e241686ddce63028cca486f4937fffad851519903da88d3?s=96&d=robohash&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/9a938d08ab9251a99e241686ddce63028cca486f4937fffad851519903da88d3?s=96&d=robohash&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/9a938d08ab9251a99e241686ddce63028cca486f4937fffad851519903da88d3?s=96&d=robohash&r=g\",\"caption\":\"Damanpreet\"},\"url\":\"https:\\\/\\\/blog.spike.sh\\\/author\\\/daman50ab08d672\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Postmortem of On-Call System Discrepancy","description":"Read the postmortem on on-call system discrepancy.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/","og_locale":"en_GB","og_type":"article","og_title":"Postmortem of On-Call System Discrepancy","og_description":"Read the postmortem on on-call system discrepancy.","og_url":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/","og_site_name":"Spike&#039;s blog","article_published_time":"2025-12-04T15:15:21+00:00","article_modified_time":"2026-01-09T07:46:39+00:00","og_image":[{"width":1040,"height":564,"url":"https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/Postmoretem.png","type":"image\/png"}],"author":"Damanpreet","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Damanpreet","Estimated reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/#article","isPartOf":{"@id":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/"},"author":{"name":"Damanpreet","@id":"https:\/\/blog.spike.sh\/#\/schema\/person\/4bc971837058eacf02e9ef12fb625155"},"headline":"Postmortem of On-Call System Discrepancy","datePublished":"2025-12-04T15:15:21+00:00","dateModified":"2026-01-09T07:46:39+00:00","mainEntityOfPage":{"@id":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/"},"wordCount":806,"commentCount":0,"image":{"@id":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/#primaryimage"},"thumbnailUrl":"https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/Postmoretem.png","articleSection":["Postmortem"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/","url":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/","name":"Postmortem of On-Call System Discrepancy","isPartOf":{"@id":"https:\/\/blog.spike.sh\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/#primaryimage"},"image":{"@id":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/#primaryimage"},"thumbnailUrl":"https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/Postmoretem.png","datePublished":"2025-12-04T15:15:21+00:00","dateModified":"2026-01-09T07:46:39+00:00","author":{"@id":"https:\/\/blog.spike.sh\/#\/schema\/person\/4bc971837058eacf02e9ef12fb625155"},"description":"Read the postmortem on on-call system discrepancy.","breadcrumb":{"@id":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/"]}]},{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/#primaryimage","url":"https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/Postmoretem.png","contentUrl":"https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/Postmoretem.png","width":1040,"height":564,"caption":"blog cover postmortem of on-call system discrepancy"},{"@type":"BreadcrumbList","@id":"https:\/\/blog.spike.sh\/postmortem-on-call-system-discrepancy\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blog.spike.sh\/"},{"@type":"ListItem","position":2,"name":"Postmortem of On-Call System Discrepancy"}]},{"@type":"WebSite","@id":"https:\/\/blog.spike.sh\/#website","url":"https:\/\/blog.spike.sh\/","name":"Spike&#039;s blog","description":"Learnings and opinions in a changing world","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.spike.sh\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"},{"@type":"Person","@id":"https:\/\/blog.spike.sh\/#\/schema\/person\/4bc971837058eacf02e9ef12fb625155","name":"Damanpreet","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/secure.gravatar.com\/avatar\/9a938d08ab9251a99e241686ddce63028cca486f4937fffad851519903da88d3?s=96&d=robohash&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/9a938d08ab9251a99e241686ddce63028cca486f4937fffad851519903da88d3?s=96&d=robohash&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9a938d08ab9251a99e241686ddce63028cca486f4937fffad851519903da88d3?s=96&d=robohash&r=g","caption":"Damanpreet"},"url":"https:\/\/blog.spike.sh\/author\/daman50ab08d672\/"}]}},"modified_by":"Damanpreet","jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/Postmoretem.png","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pfMe4Q-1aP","jetpack-related-posts":[{"id":4478,"url":"https:\/\/blog.spike.sh\/introducing-a-more-flexible-oncall-schedule\/","url_meta":{"origin":4515,"position":0},"title":"Introducing a More Flexible On-Call Schedule","author":"Kaushik","date":"3rd December, 2025","format":false,"excerpt":"We are introducing new on-call features: Add Gaps to on-call, Scheduled Layers, Handoff Days, and more.","rel":"","context":"In &quot;Announcement&quot;","block_context":{"text":"Announcement","link":"https:\/\/blog.spike.sh\/category\/announcement\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/background-46-3.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/background-46-3.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/background-46-3.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/12\/background-46-3.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":4023,"url":"https:\/\/blog.spike.sh\/jsm-review-oncall-2026\/","url_meta":{"origin":4515,"position":1},"title":"Jira Service Management (JSM) Review for On-Call Management (2026)","author":"Sreekar","date":"9th November, 2025","format":false,"excerpt":"As OpsGenie sunsets, is JSM the answer? This Jira Service Management (JSM) Review for on-call explores its features, reveals critical gaps, and introduces a more flexible, affordable alternative for your migration.","rel":"","context":"In &quot;JSM&quot;","block_context":{"text":"JSM","link":"https:\/\/blog.spike.sh\/category\/comparison\/jsm\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/11\/Basics-of-Incident-Management-13.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/11\/Basics-of-Incident-Management-13.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/11\/Basics-of-Incident-Management-13.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/11\/Basics-of-Incident-Management-13.png?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/11\/Basics-of-Incident-Management-13.png?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/11\/Basics-of-Incident-Management-13.png?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":2149,"url":"https:\/\/blog.spike.sh\/pagerduty-review-for-oncall-management\/","url_meta":{"origin":4515,"position":2},"title":"PagerDuty Review for On-Call Management (2026)","author":"Sreekar","date":"1st July, 2025","format":false,"excerpt":"If you're not sure whether to pick PagerDuty for on-call management, this review will help you decide. I signed up for PagerDuty, created and edited on-call schedules, studied on-call dashboard, and created on-call overrides. Overall, I evaluated PagerDuty's on-call management across 5 key criteria. You'll discover what these criteria are\u2026","rel":"","context":"In &quot;PagerDuty&quot;","block_context":{"text":"PagerDuty","link":"https:\/\/blog.spike.sh\/category\/comparison\/pagerduty\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/07\/PagerDuty-Review-for-On-Call-Management-2025-1.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/07\/PagerDuty-Review-for-On-Call-Management-2025-1.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/07\/PagerDuty-Review-for-On-Call-Management-2025-1.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/07\/PagerDuty-Review-for-On-Call-Management-2025-1.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":2227,"url":"https:\/\/blog.spike.sh\/6-opsgenie-alternatives-for-on-call-management\/","url_meta":{"origin":4515,"position":3},"title":"6 OpsGenie Alternatives for On-Call Management","author":"Sreekar","date":"8th July, 2025","format":false,"excerpt":"You're likely here because you heard the news: Atlassian ended new sales for OpsGenie on June 4, 2025, with a complete shutdown scheduled for April 2027. For years, OpsGenie has been the backbone of on-call management for countless teams. It might have been your team's trusted solution too. But now,\u2026","rel":"","context":"In &quot;OpsGenie&quot;","block_context":{"text":"OpsGenie","link":"https:\/\/blog.spike.sh\/category\/comparison\/opsgenie\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/07\/6-OpsGenie-Alternatives-for-On-call-Management-2025-1.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/07\/6-OpsGenie-Alternatives-for-On-call-Management-2025-1.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/07\/6-OpsGenie-Alternatives-for-On-call-Management-2025-1.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/07\/6-OpsGenie-Alternatives-for-On-call-Management-2025-1.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":1841,"url":"https:\/\/blog.spike.sh\/oncall-schedules-everything-you-need-to-know\/","url_meta":{"origin":4515,"position":4},"title":"On-Call Schedules: Everything You Need to Know","author":"Sreekar","date":"18th June, 2025","format":false,"excerpt":"I use Slack daily. It works perfectly fine. Outages rarely happen. Even if they happen, they are resolved quickly. And this is the same for many other tools. But how are they all doing it\u2014keeping services running and resolving issues quickly? Their secret lies in On-Call Schedules. On-call schedules make\u2026","rel":"","context":"In &quot;Oncall&quot;","block_context":{"text":"Oncall","link":"https:\/\/blog.spike.sh\/category\/incident-management\/oncall\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/06\/On-call-schedules-everything-you-need-to-knwo.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/06\/On-call-schedules-everything-you-need-to-knwo.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/06\/On-call-schedules-everything-you-need-to-knwo.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2025\/06\/On-call-schedules-everything-you-need-to-knwo.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":4782,"url":"https:\/\/blog.spike.sh\/things-to-do-before-you-go-oncall\/","url_meta":{"origin":4515,"position":5},"title":"5 things to do before you go on-call for the first time","author":"Sreekar","date":"19th January, 2026","format":false,"excerpt":"This guide walks you through five setup steps before your first on-call shift begins. From calendar integration to on-call notification and backup coverage, these configurations help you stay informed, organized, and supported when incidents strike.","rel":"","context":"In &quot;Guides&quot;","block_context":{"text":"Guides","link":"https:\/\/blog.spike.sh\/category\/guides\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2026\/01\/Basics-of-Incident-Management-1.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2026\/01\/Basics-of-Incident-Management-1.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2026\/01\/Basics-of-Incident-Management-1.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/blog.spike.sh\/wp-content\/uploads\/2026\/01\/Basics-of-Incident-Management-1.png?resize=700%2C400&ssl=1 2x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/blog.spike.sh\/wp-json\/wp\/v2\/posts\/4515","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.spike.sh\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.spike.sh\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.spike.sh\/wp-json\/wp\/v2\/users\/263547071"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.spike.sh\/wp-json\/wp\/v2\/comments?post=4515"}],"version-history":[{"count":5,"href":"https:\/\/blog.spike.sh\/wp-json\/wp\/v2\/posts\/4515\/revisions"}],"predecessor-version":[{"id":4525,"href":"https:\/\/blog.spike.sh\/wp-json\/wp\/v2\/posts\/4515\/revisions\/4525"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.spike.sh\/wp-json\/wp\/v2\/media\/4520"}],"wp:attachment":[{"href":"https:\/\/blog.spike.sh\/wp-json\/wp\/v2\/media?parent=4515"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.spike.sh\/wp-json\/wp\/v2\/categories?post=4515"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.spike.sh\/wp-json\/wp\/v2\/tags?post=4515"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}