Zendesk is our support queue for customer inquiries. Each case is then converted into a Zendesk ticket and filtered into a `View`, ordered by category. As part of their daily workflow, the Billing Operations Team monitor these views and process tickets through completion.
See also: [Zendesk](/handbook/security/customer-support-operations/docs/zendesk/)
See also: [Zendesk](/handbook/security/customer-support-operations/zendesk/)
@@ -100,7 +100,7 @@ Not all Fulfillment features are available at the time for all types of customer
## Internal GitLab Duo license requests
The [team member license process](/handbook/security/customer-support-operations/docs/team_member_licenses/) handled by Support Operations allows for generation of a legacy license file for team members looking to test or use GitLab Premium or Ultimate on a Self-managed instance. This license type is not compatible with GitLab Duo.
The team member license process handled by Support Operations allows for generation of a legacy license file for team members looking to test or use GitLab Premium or Ultimate on a Self-managed instance. This license type is not compatible with GitLab Duo.
- For testing of GitLab Duo, a staging cloud license can be provided to team members. Please submit a request in the Slack channel [#g_provision](https://gitlab.enterprise.slack.com/archives/C01BPM66KNE).
- In specific situations only, if a production cloud license is required for testing GitLab Duo, a $0 subscription may be created. Please reach out to the Order Management team to discuss in the Slack channel [#sales-support](https://gitlab.enterprise.slack.com/archives/CNLBL40H4).
| [Carlo Curato](https://gitlab.com/ccurato) | Director, Business Technology Operations |
| [Michael Beltran](https://gitlab.com/MBeee) | Sr. Manager, IT |
| [Jason Colyer](https://gitlab.com/jcolyer) | Fullstack Engineer, Customer Support Operations |
| [Nabeel Bilgrami](https://gitlab.com/nabeel.bilgrami) | Customer Support Operations Specialist |
@@ -34,50 +32,43 @@ The purpose of Customer Support Operations is to enable GitLab to provide deligh
Or you can reach out to us in Slack via [#support_operations](https://gitlab.enterprise.slack.com/archives/C018ZGZAMPD).
## Basic issue flow
### Bug Reports
The issue will be created in the `Triage` stage. From here, Customer Support Operations will validate the bug (if it is invalid, the request will be closed).
If it is valid, it will then move to the `Planning` stage (with all appropriate labels put in place), where a gameplan will be made.
Once a gameplan is made, it will jump to the `Development` stage, where the changes will be made.
Once it is ready for review, we will ask the requester (or someone they delegate this to) will validate the changes fix the bug.
Once validated, it will then move to the `Implementation` stage, where it will be implemented into production.
Once that is done, the issue will move to the `Completed` stage (and be closed out).
### Feature requests
The issue will be created in the `Triage` stage. From here, Customer Support Operations will determine if the request has enough information to move onto next stages (if not, we will ask for more information).
Customer Support Operations will then determine if all needed approvals are present. The general logic used is:
- If the request aligns with something on the support roadmap and seems congruent with the intention, then no approval required
- If the request aligns with something on the support roadmap but is not congruent with the intention, then we will ask for one of the following:
- The support roadmap be updated
- Support leadership approval be documented on the request
- If the request does not align with something on the support roadmap, then we will ask for an appropriate level of approval from support leadership based on:
- the amount of work
- any danger to planned work
- effect on customer or support workflows
If approved, it will then move to the `Planning` stage (with all appropriate labels put in place), where a gameplan will be made.
Once a gameplan is made and added to the issue, it will then move to the `Scheduling` stage. Here, Customer Support Operations will determine when it can be implemented. After that is decided, an iteration, milestone, and Customer Support Operations DRI will be added to the issue (which indicates the period the work will be done in).
If the iteration is not the current one, then the request will be moved to the `Queued` stage, where it will wait until the iteration the request is assigned to comes up.
Once the iteration period arrives, the request will move to the `Development` stage. Here, work will be done to get the changes into a state where they can be tested and validated. Notes will often be added on what kind of changes were made (and where) to help in later stages.
Once all changes are ready to be validated, we will add the label `Validation::Requested` and make a comment on the issue asking the the requester (or someone they delegate this to) to validate the changes done will meet their requirements for the request. What happens next depends on the results of said validation:
- If approved, the label `Validation::Received` will be added to the issue (and the issue moved to the `Implementation` stage)
- If not approved, the label `Validation::Rejected` will be added to the issue (where it will remain in `Development` to be tweaked for another round of validation).
Once a request is in the `Implementation`, we will add a comment detailing the technical blueprint of what was done (which should include links to merge requests, followup issues, etc.). We will also begin implementation the changes into production (the exact method depends on what is changing).
Once all that is done, the stage moves to `Completed`, where the issue is closed out.
## Issue flowchart
```mermaid
graph TD;
Start--> Triage;
Triage--> Type;
Type-->|Bug| BugDevelopment
Type-->|Feature| FeaturePlanning
BugDevelopment--> BugValidation
BugValidation--> BugValidated
BugValidated-->|No| BugDevelopment
BugValidated-->|Yes| Implementation
FeaturePlanning--> FeatureScheduling
FeatureScheduling--> FeatureIsItTime
FeatureIsItTime-->|No| FeatureQueued
FeatureIsItTime-->|Yes| FeatureDevelopment
FeatureDevelopment--> FeatureValidation
FeatureQueued--> FeatureIsItTime
FeatureValidation--> FeatureValidated
FeatureValidated-->|No| FeatureDevelopment
FeatureValidated-->|Yes| Implementation
Implementation--> Completion
Completion--> End
Start(Issue created)
Triage(Triage stage)
Type{Request type?}
BugDevelopment(Development stage)
BugValidation(Validation stage)
BugValidated{Was it validated?}
FeaturePlanning(Planning stage)
FeatureScheduling(Scheduling stage)
FeatureIsItTime{Is it being worked in the current iteration?}
Audits are what we call the process (and core responsibility) that involves reviewing who has what access to various platforms.
### Why do we perform audits
There are a good number of reasons that could be made for doing audits, but the biggest ones for us are:
- Ensure no security issues are occurring in the agent/user space.
- Ensure we have accurate information for procurements and renewals.
- Ensure we follow best practices for reviewing who is using the various systems we manage.
### What is audited
- Zendesk Global
- Zendesk US Government
### When are audits done
On the 1st day of each quarter, issues will be generated via the [System Audits](https://gitlab.com/gitlab-support-readiness/system-audits) project. After issue creation, audits are then performed.
## Zendesk Global audit
This is done via the `bin/zendesk_global` script. When the script runs, it does the following:
- Gathers Support team information from the Support Team YAML files
- Gathers a list of all agents in Zendesk (i.e. those with a role of `admin` or `agent`)
- Creates three lists of users:
- Light agents (those with a custom role ID of `360004984553`)
- Admins (those with a custom role ID of `360004957599`)
- Full agents (those not listed in the Light agents and Admins lists)
- Loops over the Support team information, checking:
- If they have a corresponding Zendesk user (as they should)
- If they are a light agent (as they should not be)
- If their Zendesk name matches the name for their Support team information (as it should)
- If their Zendesk email matches the name for their Support team information (as it should)
- If they incorrectly have the Zendesk administrator role (as this is closely controlled)
- Loops over all agents in Zendesk to check if any of them are suspended (as they should not be)
- Creates an issue, detailing the following
- List of all light agents
- List of all full agents
- List of all admins
- List of all failed checks
Those working the issue generated will review the issue for any problems and look into any failed checks that arose.
After rectifying any problems, the issue is then closed out.
## Zendesk US Government audit
This is done via the `bin/zendesk_us_government` script. When the script runs, it does the following:
- Gathers Support team information from the Support Team YAML files
- Gathers a list of all agents in Zendesk (i.e. those with a role of `admin` or `agent`)
- Creates three lists of users:
- Light agents (those with a custom role ID of `360008074111`)
- Admins (those with a custom role ID of `360016820032`)
- Full agents (those not listed in the Light agents and Admins lists)
- Loops over the Support team information, checking:
- If their Zendesk name matches the name for their Support team information (as it should)
- If their Zendesk email matches the name for their Support team information (as it should)
- If they incorrectly have the Zendesk administrator role (as this is closely controlled)
- Loops over all agents in Zendesk to check if any of them are suspended (as they should not be)
- Creates an issue, detailing the following
- List of all light agents
- List of all full agents
- List of all admins
- List of all failed checks
Those working the issue generated will review the issue for any problems and look into any failed checks that arose.
After rectifying any problems, the issue is then closed out.
Calendly is an online appointment scheduling tool that GitLab uses.
## Calendly webhooks
### Listing all webhooks
To do this, you will want to use the [List Webhook Subscriptions](https://developer.calendly.com/api-docs/faac832d7c57d-list-webhook-subscriptions) to get that information. This requires you to specify a scope and the organization itself. This might look like (assuming you URL encode the values, which isn't "required" but helpful):
To create a webhook in Calendly, you have to use the API.
The very *first* thing you are going to need to do is determine our Organization's information, as it will be required for the webhook's creation.
To do this, you are going to need to use the [Get current user](https://developer.calendly.com/api-docs/005832c83aeae-get-current-user) API endpoint, like so:
```bash
curl --request GET \
--url https://api.calendly.com/users/me \
--header'Authorization: Bearer TOKEN_GOES_HERE'\
--header'Content-Type: application/json'
```
This will produce output that contains our organization's information (as you are part of our organization). The exact entry you are wanting is the `current_organization` value. If using something like [jq](https://jqlang.github.io/jq/), you can run the entire command like so:
```bash
curl -ss--request GET --url https://api.calendly.com/users/me \
--header'Authorization: Bearer TOKEN_GOES_HERE'\
--header'Content-Type: application/json'\
| jq '.resource.current_organization'
```
You will also need your user reference URL, which you can get from the same endpoint. In this case, you'll need `uri` value from the output. Using something like [jq](https://jqlang.github.io/jq/), you can run the entire command like so:
```bash
curl -ss--request GET --url https://api.calendly.com/users/me \
--header'Authorization: Bearer TOKEN_GOES_HERE'\
--header'Content-Type: application/json'\
| jq '.resource.uri'
```
With that, you can now create the webhook itself. To do this, you will use the [Create Webhook Subscription](https://developer.calendly.com/api-docs/c1ddc06ce1f1b-create-webhook-subscription) API endpoint. This is going to require some very specific information to work correct, so let's break it down item by item:
-`url`
- This is the URL the payload from Calendly is being sent to.
-`events`
- An array of items the webhook will trigger for. The options currently are:
-`invitee.created`
-`invitee.canceled`
-`invitee_no_show.created`
-`routing_form_submission.created`
-`organization`
- The URL for your organization. See above to get this value.
-**NOTE** The *full* URL should be used.
-`user`
- Your user reference URL. See above to get this value
-**NOTE** The *full* URL should be used.
-`scope`
- The scope this runs on. It can either be `user` or `organization`.
Putting this all together, you will make a POST request, with the data being in JSON format. To accomplish this, we recommend making a JSON file containing the parameters to send, verifying it via [jq](https://jqlang.github.io/jq/), and then making your API call. An example of what this might look like is:
The response you get back from this need to be verified, but it should mirror a lot of the information you just used in your parameters. Once you have verified it all looks correct, the webhook is live.
- Project repo: [Calendly Events to gCal Events](https://gitlab.com/gitlab-support-readiness/calendly-events-to-gcal-events)
{{% /alert %}}
### What is Calendly Events to gCal Events
The Calendly Events to gCal Events is a setup that takes payloads from Calendly webhooks (fired at the organization level when events are created) and translates them to Google calendar entries.
### How it works
Via a Calendly webhook (implemented at the organization level), when an event is created a payload is sent to the process (via a pipeline trigger). This results in the execution of the `bin/process` script, which does the following:
- Verifies there is a payload present
- Reads data from the Support Team YAML files
- Checks if the event is a Support US Government event or a Support Global event
- It is a Support US Government event if it meets all of the following criteria:
- the assignee is a US Government Support agent
- the name of the scheduled event contains one of the following strings:
-`US Federal`
-`US Government`
- It is a Support Global event if it meets all of the following criteria:
- it is not a Support US Government event
- the name of the scheduled event contains the string `Support` (case insensitive)
- If it is a Support US Government event, a Google calendar event is added to the `US Government Support` calendar
- If it is a Support Global event, a Google calendar event is added to the `GitLab Support` calendar
- If it neither type of event, it exits with a status code of 0
### Changing Calendly Events to gCal Events
{{% alert title="Note" color="primary" %}}
- This requires at least `Developer` access to the [Calendly Events to gCal Events](https://gitlab.com/gitlab-support-readiness/calendly-events-to-gcal-events) project.
- This should only be done if there is a corresponding request issue (Feature Request, Administrative, Bug, etc.). If one does not exist, you should first create one (and let it go through the standard process before working it).
{{% /alert %}}
To make changes to the Calendly Events to gCal Events, you will need to create a MR in the project repo. The exact changes being made will depend on the request itself.
After a peer reviews and approves your MR, you can merge the MR (which will have them applied on the next scheduled run).
## Common issues and troubleshooting
This is a living section that will have items added to it as needed.
