Identity Vision Document

A note on contributing to this document: Any member of the community is welcome to contribute to this document by opening a pull request on GitHub. Please include your rationale and any context or pitch for your idea in the contents of the pull request, and we can discuss your ideas there, and make amendments directly on that branch.

Please write goals as top-level bullet points, and potential features as nested points, e.g.:

• Make it easy for campaigners to send great emails (goal)
  • Have pretty buttons that take people to action pages (feature)
  • Paperclip helper helps you write your email (feature)

The prompt: What should Identity look like / act like in 2-3 years? What do we want to be true about the application and/or its peripherals, collaboration, and ecosystem?

Feature Goals

An industry-leading mailer

• UX that’s well optimised for our use-cases (rapid-response campaigns, higher-touch organising for small teams)
  • Mailing HTML is highly component-ised with MJML for responsive mailing elements that look consistent across devices and are easy for campaigners to use
  • Live preview feature that is accurate and responsive
  • Errors (misspelled merge tags, test case missing, HTML dropped into the middle of a merge tag) are always reported to the user with a clear description of the problem and a hint for resolving it.
  • Mailer editing that works well for distributed/remote teams
    • Real-time collaboration, or a way to handle locks so people don’t overwrite each other
    • Comment threads in the mailer interface
    • Easy to paste from gdocs
    • Some way of handling “assign this email to X for review / user-journey proofing / editing / writing”
  • Giphy integration in the mailer
• Sending blasts is FAST and STABLE – 1 million mailings per hour or better
  • Definitely aiming for ‘better’ here; 38 Degrees is already sending 1m/hour
  • Caching of lists and list members, and mailings and mailing_varations
  • This (hopefully) fits in nicely to API/comms between Core and Mailer
  • Good code coverage
  • Solid benchmarking that we run regularly
• Sophisticated A/B/X testing
  • Test different “voices” or “themes” as well as individual bits of content
  • Long-term testing, separating users into stable group to run tests e.g. on branding updates or new best practices
  • Test components in a way that’s invisible to campaigners
  • Possible to test so that test groups remain stable across multiple content blocks, so e.g. mailing is split into “urgent” vs “inspiring” and the subject, featured image, and the ask can all change to be either “double the urgency” or “double the inspiration”, testing different voices or themes throughout a mailing
  • Test whole mailings against one another under a more formal data object like “mailing group” to formally link different mailings that are tests of the same thing
  • The interface to see the test results makes it easy and fast to make decisions based on those results
• Industry-leading email delivery rates:
  • Flawless handling of things like SPF, DKIM, DMARC, and alerts to admins when these are producing warnings or errors
  • Mainstream the “respond to this email to sign the petition” workflow for better inbox placement
  • Integrated Litmus testing (or similar product solution)

Fun and convenient to use

• An interface for campaigners to guess which subject line will win the test before you send it; everyone guesses and then after the results are stable you can see how your guesses turned out. Helps campaigners think about good subject lines and improve their skills over time.
• Workflow integrations for things like Slack, Trello
• Auth system integrates well with Google Auth, Authy, other systems as needed
• Possibility to merge members. If a person exists in Identity with 2 or more different email addresses, with actions split between the different emailadresses it is difficult to get an overview of a persons activities. The same goes for other data that is divided between the different adresses (phonenumbers, adresses etc.). There appears to be a mergefunctionality but there is no UI available.

A strong core that integrates services

• Better integrations of census data (like augmented datasets)
• FALC – fully automated luxury campaigning
  • A/B tests pick their own winners and keep sending and narrowing options to get to a winner quickly without using as much list / campaigner brainspace.
  • Use machine learning and language generation to write tweets or text messages or email drafts that humans would then review and send
  • (Actually James has already prototyped this and it’s… freakishly good.)
• Broadcast channel diversity! Email, SMS, Web Push, WhatsApp, Telegram, etc. Identity can send blasts to lots of them, and has intelligent ways of optimising how we communicate with different people based on their usage and responsiveness
• Mailer and Lister enable campaigners to use all the rich data available in the Core
  • Each email’s/list’s benchmarks are available from the time of sending (expected open, click, actions, $)
• Able to process qualitative data (like from Surveys) to personalize communications and make people feel like a part of the org
  • Typeform integration (already done)

Other modules

• IRL volunteer management CRM attached, i.e. “Election Mode”
• Shortlinks manager
• Blast communications facilitate or lead into quality 1-to-1 interactions (like Spoke x email)
  • The “reply to sign” feature
• Incoming data ingestion is stable and scales to meet basically any situation
  • Rewrite CSL integration to ingest their Webhooks instead of CSVs

Communications Traffic management

• An overview interface that allows you to see the upcoming week’s email calendar
• Email priority levels to resolve targeting overlaps (and make it easier to schedule “lazily” without worrying about overlaps)
• Easy to “clean up” the next day or later in the week, sending emails to the remainder of eligible members who were excluded because of targeting conflicts
• Omni-cast communication flows (like Mautic) that respond to users interactions and preferences to move them along a user journey
• Enter an email flow, such as a launch email, kicker to non-signers, upsell to regular donors who sign, and followup share ask to non-donors who sign, and then after the launch you can fill in the contents of those followup emails and the target-and-send is fairly
• Other pings like sending an SMS to certain people to bump an email like an event invite, composed and scheduled along with the original email.
• Specific details of list-building are handled by machines:
  • e.g. “probabilistically prioritise this mailing”,
  • e.g. “This user is on the list for both our Enviro email and our Corruption email today, but they should only get one of them. The environment mailing is performing 20% better than its recipients’ expected responsiveness, and the corruption mailing is performing 10% worse than expected, and this member has a corruption responsiveness score of 0.7 and an environment responsiveness score of 0.4, so the system will know to send them the Enviro email today.”
  • e.g. “List cleanup” features like building a target list with 120k people, and being able to test 20k and then pick the winner and have it send to the other 100k without having to manually create a bunch of lists

Anti-Goals

• No blockchain

General Philosophy of the Code Base:

• Loose coupling of functionality, and solid definitions of the different problem domains.
• Bulk database interactions happen asynchronously when possible, using lots of Sidekiq a lot (e.g., recording an action, an open, or a click, updating member details from an action taken on an external service)
• Certain actions, like updating member details and recording actions often include sensitive information like consents, and difficult to parse information like addresses, so we have two methods, UpsertMember#call and Member#record_action, which we think get these really right and we re-use them all over the place.
• Every org feels confident pulling from master frequently, because they have either written or checked over tests of their critical functionality.
• Modularity: broadcast communicators, shortlinks, analytics, member portal, and CRM workflows – all of these problem domains have solutions with independent modules or code bases
• Avoid feature creep; each component should “do one thing and do it well”
• Avoid tightly-coupling components and other peripheral applications; use clean APIs, not direct DB access

The Collaboration and Community

• Some way for smaller teams to propose good ideas and have others agree and pitch in resources or time to bring them to completion.
• Pull Requests are reviewed promptly and kindly. Where there are disagreements on approach there is a clear process for resolving them.
• There is a clear path for becoming “an identity maintainer”, where that means someone comfortable / mandated to merge and/or push to master. (e.g., has made at least X PRs, had a chat with another maintainer for handover of standard merge practices).