AI-Generated Summary Now Supports Follow-Up Questions
The “Switch” AI chatbot in Harness Feature Management and Experimentation (FME) has been renamed to “Release Agent” and now supports follow-up questions after you click “Summarize” in metric details.
To see the Metric summary and ask follow-up questions:
Drill into a metric tile on a Metrics impact dashboard and click Summarize.
After viewing the summary, type your follow-up question and click Continue conversation in Release Agent.
Continue to ask additional follow-up questions if you would like, including suggestions for next steps.
Note: the transition from “Switch” to “Release Agent” will take place gradually. For now, you’ll still see Ask Switch in the lower left navigation of Harness Feature Management and Experimentation:
Harness Feature Management and Experimentation (FME) now supports “Large segments” (lists of targeting IDs) that can contain more than 100,000 IDs.
Large segments support multiple use cases where bulk targeting of specific IDs is required:
Communicating with more than 100,000 specific customers in-app after an incident.
Targeting any set of users based on attributes not available within the app at runtime.
Performing large-scale A/B tests on specific user bases, exported from external tools.
Effective immediately, Enterprise tier customers may create and use Large segments containing up to one million (1,000,000) IDs. Significantly higher limits are available by request.
Learn more about Large segments and the ways they differ from Standard segments in the documentation:
Note: The initial release of Large segments is focused on client-side SDK usage only. Server-side SDKs do not yet support Large segments, but soon will. Until they are supported, evaluations of feature flags that target Large segments will return control on server-side SDKs.
Admin API Endpoints
After familiarizing yourself with Large segments at the above links, you may find these UI and API equivalent documentation links handy for automating the steps via the Admin API:
Steps for creating and populating a Large segment using either UI or API
Create a Large segment (just metadata, no Environment definition)
Semantic Versioning (SemVer) Attribute Dictionary Support
Split FME now supports SemVer type attributes and suggested values in the attribute dictionary:
Admins can create SemVer typed attribute names and suggested values.
Users can see the SemVer attribute names and suggested values when editing targeting rules.
Attribute dictionary support reduces guesswork and manual errors when editing targeting rules.
What you need to know in a nutshell:
The semver standard calls for versions to be formatted as major.minor.patch
Split FME added SemVer support on June 6th, 2024 , eliminating the need to write regular expressions (i.e. regex) to target version ranges or “is a version less than or equal to x.y.z”
This update adds the benefits of standardized attribute names and suggested values delivered by our Attribute Dictionary.
What SemVer attribute support looks like to an admin:
Admins can create a custom attribute of type “Semver” and optionally enter suggested values.
What SemVer attribute support looks like to a user:
Choose an attribute name from the attribute dictionary, such as “ios_version”
If the chosen attribute is of type SemVer, the appropriate matchers are shown:
If “is in list” is chosen as the matcher type, suggested values are shown:
Guardrail and Key Metric Alerts Now Shown in the Alerts Table
Previously, the Alerts table on the Monitoring tab displayed Metric alerts only. Guardrail Metric alerts and Key Metric alerts generated email notifications to feature flag owners, but were not persisted in the UI.
Now all three types of alerts are shown on each flag’s Monitoring tab for any team member to see. The table has been simplified to display the most valuable fields at a glance, reducing cognitive load. Details less critical for quick triage remain available under an info icon.
As a refresher, here is quick summary of the three alert types:
Metric Alert
Triggered: when the relative or absolute impact threshold in an Alert policy is reached
Monitors: percentage rollouts only for flags where the metric is a Key metric
Release date:
2024-09-30
Better Together: Split + Harness
New Colors, Names for Organization and Workspace
Starting on September 30th, we began a progressive rollout to update the Split UI, bringing it closer to the look of Harness:
Beyond a change in color scheme, you will also see two changes to terminology:
Workspaces will now be known as Projects in the UI
Organizations will now be known as Accounts in the UI
Note: These terminology changes are being made only to labels in the UI at this time. To avoid introducing a breaking change, the Admin API will continue to use the strings ws, workspace, organizationId, and orgId until further notice.
Release date:
2024-09-12
Monitoring
Traffic Insights and Alerts
The Alerts tab has been renamed Monitoring and expanded to show real-time traffic insights over time and any alerts fired for the flag on a single page.
By default, traffic over the Last 48 hours is shown, but you may also select the Last 7 days, another Time range, or a specific Feature flag version:
Changes made to flags (i.e., new flag versions) are displayed as vertical bars for context:
When requesting a manual recalculation of metrics, you can reduce the computation time by limiting the recalculation to Key metrics. To select this faster path to results, click the Recalculate button and then click Key metrics only. If you prefer to recalculate Key, Supporting, and Guardrail metrics, click All metrics instead.
Reminder: Key, Supporting, and Guardrail metrics are automatically calculated on a schedule, starting five minutes after a feature flag change and going to progressively longer intervals until the last automated calculation at 28 days. You do not need to press the Recalculate button unless you want to kick off a calculation before the next scheduled run or after the 28-day mark.
We’re excited to announce the next step in improving metric categorization for your feature releases and experiments: Supporting metrics. This new metric category gives you greater control over metrics monitored per feature flag, helping you focus on what truly matters.
What’s changing?
On the Metrics impact page, we have replaced “Organizational metrics” with “Supporting metrics.” Now, you can easily select specific additional metrics to track for each release, reducing noise and making it easier to analyze your results.
How do I assign flag-specific metrics?
Key metrics and Supporting metrics are specific to individual feature flags, and can be managed on a flag’s Metrics impact tab. Under each category, click “Add metric” to initiate the process of assigning metrics to the category. Once metrics are added, you can wait until the next scheduled calculation or manually “Recalculate metrics” to see results.
How do I ensure my important metrics are still monitored for every flag?
In June, Split introduced Guardrail metrics to improve your ability to define which metrics should be monitored for every flag. Guardrail metrics include automated alerting, meaning flag owners will be notified as their releases or experiments impact these metrics.
Since Organizational metrics will no longer be available, we recommend adding important metrics to the new Guardrail metrics category ASAP to ensure they continue to be protected and limit any disruption of analyzing metric results. Guardrail metrics can be assigned in the metric definition:
The combination of Key metrics, Guardrail metrics, and Supporting metrics will reduce noise and increase sensitivity while ensuring important metrics are monitored for every feature release or experiment. We welcome your feedback as we continue to improve our metric results!
Release date:
2024-09-04
Monitoring
Guardrail Metric Alerts
Flag owners will now automatically receive alerts on any guardrail metric without manual configuration.
Alerts are sent to flag owners whenever a guardrail metric moves significantly in the desired or undesired direction. This feature is designed to help you make safe and accurate release decisions in a timely manner.
As a reminder, Guardrail Metrics (released 2024-06-14) ensure that an organization’s most crucial metrics are protected and monitored throughout every feature release and experiment by making their calculation automatic and mandatory for all flags that use the same traffic type as the metric.
Metrics are set as guardrail metrics for your workspace on the Metric definition page. View the docs here
Release date:
2024-06-14
Monitoring
Guardrail Metrics
Users can now categorize their organization’s most important metrics as “guardrails”. Guardrail metrics are those your organization wants to protect during a feature release or experiment. Metrics are set as guardrail metrics for your workspace on the Metric definition page.
Release date:
2024-06-06
SDK Enhancements
Semantic Versioning Targeting
Using the latest Split SDKs, users can more easily define targeting rules for new features based on app, OS, and other versions using attribute-based targeting. The SDK then automatically serves the appropriate treatment to users without needing additional code configurations. Split’s native Semantic Versioning Targeting removes the additional complexities and manual work that comes with targeting different application versions, allowing users to seamlessly deliver different experiences.
Release date:
2024-05-31
Usability Enhancements
Left Navigation Enhancements
Left-hand navigation within the application has been optimized. With this change, we’ve narrowed the navigation bar, migrated the search function to a modal dialog, and moved account settings to the bottom of the page. This makes frequently used actions (e.g. workspace switching) more readily accessible within the UI.
Release date:
2024-05-03
SDK Enhancements
Split Suite, iOS SDK
Users can automatically capture event data in Split using their iOS SDK without needing additional agents, integrations, or track calls. This eliminates the manual process of sending events to Split, allowing users to quickly set up metrics and alert policies.
Monitoring
Out-of-the-Box Metrics
Split automatically creates metrics for any events being auto-captured by the Split Suite and RUM Agents (Web, Android, iOS). This reduces the manual effort of creating metrics and allows users to easily calculate their engineering and performance metrics.
Release date:
2024-04-16
Usability Enhancements
Switch Updates
Users can now use Switch, Split’s in-app AI assistant, to easily summarize their experimentation results. Simply click on any metric card and hit the summarize button for a full analysis of your data. Please note, that this is a Generative AI feature that leverages end-user/customer data and will only be available to users who have specifically requested it to be enabled via Split’s Support team.
Release date:
2024-02-16
Usability Enhancements
Change Request Management
To help teams easily coordinate and collaborate on flag updates, we added shareable direct links to change requests. This allows teams to quickly share feature flag updates with key stakeholders and get faster approvals when needed.
