Data Playbook for Developer Tools

Goals

• Collect all essential metrics to enable informed decision-making.
• Ensure accessibility of metrics across teams.
• Consolidate all key metrics into one dashboard for easy reference.

Actions to Take

• Review data gaps and fix bugs related to tracking.
• Update relevant software tools to their latest versions.
• Ensure consistent data flows across platforms like PostHog and HubSpot.
• Regularly audit metrics and update dashboards to reflect accurate data.

Metrics to Track

Activation Metrics:

• Number of sign-ups from within the Ideal Customer Profile (ICP).
• Website visitor engagement and conversion rates from ICP.
• Most-used cloud providers by users.

Usage Metrics:

• Number of architecture updates.
• Number of workflows started and completed.
• Number of active users, calculated on a weekly and monthly basis.
• Stickiness metrics: measure how often users return and engage over a specified time frame.

Retention Metrics:

• Weekly and monthly active users.
• Percentage of users who perform core actions like architecture updates, deployments, and versioning.
• Monitor retention rates across time periods (e.g., Week 1 to Week 6).

How to Measure

1. Product Usage:
• Define what actions count as "active usage" (e.g., updating architecture, starting workflows, planning or applying infrastructure changes).
• Use PostHog to track user behavior and identify which actions users take frequently.
2. Retention:
• Track weekly and monthly retention rates based on user activity.
• Measure drop-off points and identify where users disengage.
3. Website Traffic:
• Use website analytics to segment visitors based on intent and behavior (e.g., high-quality traffic, medium-quality traffic, low-quality traffic).
• Qualitative insights from visitor reports can help prioritize outreach and refine targeting.

Growth Funnel Metrics

1. Awareness:
• Track metrics like website visitors, sign-ups, and engagement with educational resources.
2. Acquisition:
• Monitor conversion rates from website visitors to active users.
• Capture ICP-specific metrics (e.g., sign-ups, website visits from key regions or personas).
3. Activation:
• Measure how quickly new users engage with core features like creating architectures or importing existing ones.
4. Retention:
• Focus on tracking user activity over time, especially beyond the initial onboarding period (e.g., 21 days post-sign-up).
5. Revenue:
• Measure conversion rates from free to paid users.
• Track the performance of different billing plans and how user activity correlates with plan upgrades.
6. Referral:
• Monitor how often users refer others and the success rate of referral programs.

Best Practices

• Always add detailed observations to metrics and data dashboards to facilitate continuous improvement.
• Establish regular reviews of data collection methods to ensure accuracy.
• Continuously update retention and activation metrics based on evolving user behavior.

Resources

• Regularly update and refine data dashboards using tools like PostHog and HubSpot.
• Ensure that all teams have access to real-time data for decision-making.
• Use growth funnel tracking to adjust strategies based on user engagement and retention trends.