Migrating to Stripe

Posted on December 06, 2020 · 14 mins read

NOTE: Although I am a Stripe employee, this is my personal blog. None of what I will describe has an official recommendation from Stripe. Additionally, if you follow me down this migration path, ensure that you do so in a manner that is compliant with the PCI Data Security Standard (PCI DSS).

In my last post I provided an overview of the changes I’ve made to mitcnc.org. This post details the work done to migrate our annual memberships from iModules+Cybersource to WordPress+Stripe.

Prior to the migration I wrote a WordPress Stripe plugin to integrate with Checkout and the customer portal, and render a membership level page. Our membership levels (e.g., Sustaining for $50, or Patron for $100) are pretty straightforward. Each level is modeled as a product with at least one price. The WordPress plugin allows us to determine which prices will be displayed to members at checkout time. A separate plugin—that is more specific to the needs of MITCNC, so not part of the plugin linked above—utilizes filters from the Stripe plugin to apply a discount for recent graduates, and render a lifetime membership level for members of the Cardinal & Gray Society (graduates of 50+ years).

The early investment in the plugin was key to getting support from MITCNC leadership and MITAA staff for this migration. A demo on our staging site provided a tangible experience these stakeholders could play with and compare to the iModules experience. Since I relied on Stripe to handle much of the UX, I only had to focus on building a single page where new members can select a membership level, or existing members can click through to the customer portal.

Writing the code and modeling the memberships was the easy part! Migrating card data from Cybersource to Stripe was far more difficult. I inquired within Stripe to learn more about what I should expect from the migration process. I was disappointed, but not surprised, to learn that Cybersource tends not to play nicely when users want to migrate to a different payment processor. They take a long time to send data, and the data is often incomplete. I opened support requests with Cybersource and further learned the process would take six to eight weeks and cost $500. Given the uncertainty of this process, I exercised an option I was surprised Cybersource offered: transferring unmasked account numbers via API.

Cybersource gave us the option to retrieve account numbers via API. Stripe gave us the option to transmit account numbers via API. After weighing the risks (e.g., a malicious actor gets access to the card numbers) against our need to maintain revenue and offer a better user experience, we decided to proceed with this option.

My one rule for this migration was: account numbers are never stored to disk. No CSVs. No databases. Card numbers are read from Cybersource’s API and immediately transmitted via Stripe’s API (both utilizing TLS). My laptop was physically secure, so there was minimal risk of someone (a) knowing I was running the migration script, (b) breaking into my lobby and elevator, (c) breaking down my apartment door, and (d) freezing my laptop so they could read the DRAM contents and steal a few account numbers. (Note: If you are that desperate for money, just ask for some. I am happy to give you some money and spare the insurance deductible.)

I wish I could say the migration script was as simple as making a couple API calls. Unfortunately Cybersource doesn’t make it as simple to get subscription data. Stripe has a list endpoint that allows me to retrieve a paginated list of all subscriptions. Easy. Cybersource doesn’t have such an API, and requires two separate identifiers—merchant reference code and subscription ID—to retrieve a single subscription. Neither of these identifiers is available in the data I am able to export from iModules, but they are available in daily Cybersource transaction reports.

The first script I wrote downloaded a year of daily transaction reports by making one API call per day. I distilled the logs solely to those associated with subscriptions, filtering out one-off transactions, and stored the data as both a CSV and in a SQLite database. It is important to note that this data did not include account data, hence my willingness to store it on disk. I originally stored the data solely as a CSV that was later loaded by a separate migration script. However, I later realized that I needed the ability to transact on the data to group related transactions (e.g., a member renewed their membership mid-year), and to keep track of which subscriptions had been migrated. I opted to use SQLite for this purpose since my dataset was small (<2000 rows).

The actual migration script is pretty straightforward:

  1. Read a row from the database.
  2. Pull the account data from Cybersource.
  3. Determine the Stripe product/price based on the subscription amount.
  4. Get, or create, a customer at Stripe using the email address as a primary key.
  5. Get, or create, a payment method at Stripe using the card brand and last four digits as a compound primary key.
  6. Get, or create, the subscription from Stripe using the Cybersource subscription ID (stored in Stripe metadata) as a primary key.
  7. Write the Stripe IDs to the database row.
  8. Cancel the Cybersource subscription.

Since the members had already paid for their memberships, the Stripe subscriptions were created with trials that ended at the next rollover point. This ensured members were not double-charged. In hindsight, I could have achieved the same result by initially setting the subscription to send an invoice (instead of charge automatically), immediately mark the invoice as paid, and updating the subscription to charge automatically. However, this extra effort would not have served much of a purpose for us since we don’t normally rely on trials. If your business model relies on trials, and you want the Stripe dashboard conversion/retention metrics to be accurate, I recommend the multi-step approach over trials.

The script was initially run against Stripe’s test environment, revealing a few issues I needed to resolve:

  1. Expired cards for active subscriptions. If a card expired before it could be migrated to Stripe, the script creates the subscription anyway. The member will be notified when the subscription renewal is approaching, and can update payment details before then.
  2. Multi-year subscriptions. Some of our members managed to pay for multiple years of membership in advance, resulting in a need for trials that ended well into 2023. Stripe only supports trials up to two years in length. Given we had about 10-15 such memberships, I opted to resolve them manually by setting up subscription schedules in the Stripe Dashboard. At the end of two years, another trial will be created for the remainder of the pre-paid period. After that time, the subscription will cycle to a paid subscription like the others.

We have less than 2000 subscriptions, so the bulk of the migration was done in a few hours. I spent a few more days, off-and-on, tracking down one-off issues and re-running the scripts to migrate subscriptions that were created between my initial data download and disabling the old sign-up form. This is where I learned an interesting fact: Cybersource includes read transactions in their daily transaction logs. If you happen to load 2000 subscriptions via API one day, a record of each of the 2000 API calls will be in the transaction logs for that day. Subsequent downloads of the subscription metadata took much longer than the initial one!

The migration scripts are available on GitLab. They are still somewhat hard-coded for MITCNC purposes, but should serve as a decent starting point for others. This work is available without warranty, and is not endorsed by MIT or Stripe. As stated at the beginning of this post, make sure you understand the risks before you attempt a similar migration.

Lessons learned

Some of these are just good habits. Others are mistakes I made with this migration.

Utilize the test environment. Stripe’s test environment made it easy for me to test my code without fear accidentally charging a member. I did have to modify my script to create payment methods with the test card number, however, since the test environment does not accept real card numbers.

**Build a prototype/MVP. **A couple hours on a decent demo can save even more time in back-and-forth emails and meetings, and help gain support for a migration by showing stakeholders some of the benefits of the migration.

Work with stakeholders to ensure you understand all subscription types/behaviors. This is where I messed up. I knew about the different levels, free lifetime memberships, and discounts for students. I did not know that we also offer a discount for alumni who are current students at other institutions. Modeling this is pretty straightforward, but our administrator will need to manually create these subscriptions in the Stripe dashboard until I update our plugin(s) to handle these discounts.

Consolidate your data sources. I had a record of memberships exported from iModules to my data warehouse in BigQuery. This table included one-time payments (e.g., non-renewing membership), subscriptions, and manually-created student and lifetime memberships. I should have used this one table as my source of truth, and written a single script to migrate subscriptions. Instead I was laser focused on recurring memberships, and ended up writing one script that dealt with Cybersource subscriptions, one for lifetime memberships, and another for non-recurring memberships. Additionally, I spent a bit of time syncing user data back to WordPress after the migration when it should have been done first.

If I were to repeat this procedure, I would do the following:

  1. Ensure all users are recorded in our authentication service—the WordPress user table.
  2. Sync the users to Stripe customers.
  3. Sync WordPress, Stripe, and iModules data to the data warehouse—BigQuery.
  4. Write a script that pulls Cybersource metadata—NOT card numbers— and pushes it to the data warehouse.
  5. Write a script that queries the data warehouse, pulls card numbers from Cybersource (if recurring), and creates Stripe subscriptions.

What’s next?

The migration clearly wasn’t as smooth as it could have been, but it was completed without any outages or loss of functionality for our members. We have seen subscriptions rollover and payments completed. Members are also making use of Checkout for new memberships. We still see some payment failures, but Stripe gives us more data on these failures than we received from Cybersource.

We still have some work to do to better handle MIT10 memberships for alumni who graduated in the past 10 years. These memberships have a lower price—$30 annually vs. $50—so a discount needs to be applied for the correct number of years based on the member’s graduation year. This can be done fairly easily with subscription schedules, but that functionality is not yet integrated with Checkout so I’ll need a little more connective glue to ensure a good user experience.

My series of posts on mitcnc.org continues. I plan to talk more about our cloud infrastructure and WordPress configuration in the next post. This may also include a deep dive into our data warehouse, if I don’t get too loquacious; otherwise, that may end up being its own post.

As always, if you have questions, feel free to drop me a line.