Allow to export and edit dashboards from Serverless environments by mrodm · Pull Request #3007 · elastic/elastic-package

mrodm · 2025-10-20T14:30:40Z

This PR allows to export dashboards in Serverless environments (elastic-package export dashboards command).

For Serverless environments, a new set of APIs from Elastic Package Manager have been used:

Get the current packages installed: https://www.elastic.co/docs/api/doc/kibana/operation/operation-get-fleet-epm-packages-installed
Get the information about those packages (containing asset IDs): https://www.elastic.co/docs/api/doc/kibana/operation/operation-get-fleet-epm-packages-pkgname-pkgversion
Get the actual description of those assets via https://www.elastic.co/docs/api/doc/kibana/operation/operation-post-fleet-epm-bulk-assets

If this command is used in local stacks, the same behavior is kept. Given that, all dashboards are shown to select which ones to export.

Same changes are applied to the elastic-package edit dashboards command.

This new UX is also applied to Elastic stacks >= 9.0.0.

Author's checklist

Check elastic-package export dashboards in Serverless
Check elastic-package export dashboards in local stacks (same behavior).
Check elastic-package edit dashboards in Serverless
Check elastic-package edit dashboards in local stacks (same behavior).

Both commands require the same changes to list dashboards.

Example of usage

First step all installed packages are shown:

 $ elastic-package -C ../integrations-main/packages/nginx export dashboards 
Export Kibana dashboards
Which packages would you like to export dashboards from? (nginx-2.3.2)
  elastic_package_registry-0.3.1
  fleet_server-1.6.0            
> nginx-2.3.2                   
  synthetics-1.4.2              
  The dashboard is not part of a package (show all available dashboards) 
                                

enter continue • ctrl+c cancel

If the user selects a package, it will be shown all the dashboards related to that package (e.g. nginx):

 $ elastic-package -C ../integrations-main/packages/nginx export dashboards 
Export Kibana dashboards
? Which packages would you like to export dashboards from?: nginx-2.3.2
Which dashboards would you like to export?
  Use ↑↓ to navigate, space to toggle selection, enter to confirm
> [ ] [Logs Nginx] Access and error logs (ID: nginx-046212a0-a2a1-11e7-928f-5dbe6f6f5519, Type: dashboard)
  [ ] [Logs Nginx] Overview (ID: nginx-55a9e6e0-a29e-11e7-928f-5dbe6f6f5519, Type: dashboard)             
  [ ] [Metrics Nginx] Overview (ID: nginx-023d2930-f1a5-11e7-a9ef-93c69af7b129, Type: dashboard)          

enter continue • ctrl+c cancel

If the user is developing a new dashboard (and therefore not related to any package), they should choose New dashboard option, and all the dashboards available in the Serverless instance will be shown:

 $ elastic-package -C ../integrations-main/packages/nginx export dashboards 
Export Kibana dashboards
Which packages would you like to export dashboards from? (nginx-2.3.2)
  elastic_package_registry-0.3.1
  fleet_server-1.6.0            
  nginx-2.3.2                   
  synthetics-1.4.2              
> The dashboard is not part of a package (show all available dashboards) 
                                

enter continue • ctrl+c cancel

# After choosing "New dashboard" option

 $ epd -C ../integrations-main/packages/nginx export dashboards 
Export Kibana dashboards
? Which packages would you like to export dashboards from?: The dashboard is not part of a package (show all available dashboards)
Which dashboards would you like to export?
  Use ↑↓ to navigate, space to toggle selection, enter to confirm
> [ ] [EPR Metrics] Overview (ID: elastic_package_registry-313c2700-099b-11ed-91b6-3b1f9c2b2771)
  [ ] [Logs Nginx] Access and error logs (ID: nginx-046212a0-a2a1-11e7-928f-5dbe6f6f5519)       
  [ ] [Logs Nginx] Overview (ID: nginx-55a9e6e0-a29e-11e7-928f-5dbe6f6f5519)                    
  [ ] [Metrics Nginx] Overview (ID: nginx-023d2930-f1a5-11e7-a9ef-93c69af7b129)                 

enter continue • ctrl+c cancel

How to test this PR locally

Start a new elastic-package with the serverless provider as detailed in https://github.com/elastic/elastic-package/blob/main/docs/howto/use_serverless_stack.md#the-serverless-stack-provider
Install a package and run export dashboards

cd /path/to/package
elastic-package install -v

elastic-package export dashboards -v
elastic-package export dashboards -v --id <some_dashboard_id>

elastic-package edit dashboards -v
elastic-package edit dashboards -v --id <some_dashboard_id>

Extract function to select dashboard IDs

mrodm · 2025-10-20T14:46:52Z

internal/packages/assets.go

+	// These fields are not expected to be part of responses from APIs
+	DataStream string `json:"-"`
+	SourcePath string `json:"-"`


I could not find any usage of these two fields in APIs.
It looks like that they are used in test asset and they are filled programmatically.

elastic-package/internal/packages/assets.go

Line 139 in fc03bdf

DataStream: dsManifest.Name,

elastic-package/internal/packages/assets.go

Line 171 in fc03bdf

DataStream: dsManifest.Name,

elastic-package/internal/packages/assets.go

Line 207 in fc03bdf

SourcePath: assetPath,

If the json:"-" tag is not added to the fields, queries with this struct fail with the error of those fields being not available/undefined.

mrodm · 2025-10-20T16:46:18Z

internal/kibana/dashboards.go

+func (c *Client) exportAllDashboards(ctx context.Context) ([]common.MapStr, error) {
+	logger.Debug("Export dashboards using the Kibana Saved Objects Export API")
+
+	request := ExportSavedObjectsRequest{
+		ExcludeExportDetails:  true,
+		IncludeReferencesDeep: false,
+		Type:                  "dashboard",
+	}
+
+	return c.ExportSavedObjects(ctx, request)
+}


This method would return all the dashboards available in the Kibana Instance.

According to the documentation, the number of assets that will be exported depend on savedObjects.maxImportExportSize setting (link):

NOTE: The savedObjects.maxImportExportSize configuration setting limits the number of saved objects which may be exported.

By default that setting is 10000 elements (savedObjects.maxImportExportSize setting):

The maximum count of saved objects that can be imported or exported. This setting exists to prevent the Kibana server from running out of memory when handling large numbers of saved objects. It is recommended to only raise this setting if you are confident your server can hold this many objects in memory. Default: 10000

10K sounds like enough 🙂

mrodm · 2025-10-21T07:53:14Z

/test

mrodm · 2025-10-21T13:30:48Z

internal/kibana/savedobjects.go

+	// Even if no error is returned, we need to check if the import was successful.
+	if !resp.Success {
+		return fmt.Errorf("importing %s %s was not successful", savedObjectType, id)
+	}
+


Added a new check to ensure that the saved object has been imported successfully:
https://www.elastic.co/docs/api/doc/kibana/operation/operation-post-saved-objects-import

success: Indicates when the import was successfully completed. When set to false, some objects may not have been created.

cmd/export_dashboards.go

mrodm · 2025-10-21T16:06:45Z

cmd/edit.go

+		// Not mandatory to get the package name here, but it would be helpful for users
+		// to select by default the package where they are located if any.
+		defaultPackage := ""
+		packageRoot, err := packages.MustFindPackageRoot()
+		if err == nil {
+			m, err := packages.ReadPackageManifestFromPackageRoot(packageRoot)
+			if err != nil {
+				return fmt.Errorf("reading package manifest failed (path: %s): %w", packageRoot, err)
+			}
+			defaultPackage = m.Name
+		}


I think it could be helpful for the user to select the package where they are located if any. But this should not raise an error if the command does not run from a package.

mrodm · 2025-10-22T08:07:41Z

test integrations

elastic-vault-github-plugin-prod · 2025-10-22T08:14:19Z

Created or updated PR in integrations repository to test this version. Check elastic/integrations#15710

jsoriano

Nice, works great. Thanks!

Added some comments but mostly nitpicking and suggestions, so they could be ignored 🙂 so approving by now.

cmd/export_dashboards.go

internal/kibana/savedobjects.go

jsoriano · 2025-10-22T08:36:12Z

internal/kibana/savedobjects.go

+	dashboardTitle, ok := attributes["title"].(string)
+	if !ok {
+		return DashboardSavedObject{}, fmt.Errorf("dashboard object does not contain attributes.title field")
+	}


Nit. Maybe it would be easier to use unmarshal? Though for that exportAllData would need to return the raw responses.

internal/kibana/savedobjects.go

cmd/export_dashboards.go

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

elasticmachine · 2025-10-22T12:09:52Z

💚 Build Succeeded

Buildkite Build
Commit: 9d9c0e2

History

💔 Build #6413 failed 9d4417a
💚 Build #6412 succeeded 0609cae
💛 Build #6395 was flaky 9cb72c3
💔 Build #6393 failed a8b59fb
💔 Build #6387 failed 6542265
💔 Build #6371 failed 7d6211b

cc @mrodm

jsoriano

Nice.

mrodm added 4 commits October 17, 2025 19:08

Export dashboards using EPM API endpoints

dc54003

Just execute the new method on Serverless

2fa7ec3

Select package from current directory as default

66a1770

Use same method to find dashboards in Serverless via export API

54be65a

Extract function to select dashboard IDs

mrodm self-assigned this Oct 20, 2025

mrodm commented Oct 20, 2025

View reviewed changes

mrodm added 2 commits October 20, 2025 17:38

Increase number of packages per page

d843e20

Remove Response field in response - not present

7d6211b

mrodm commented Oct 20, 2025

View reviewed changes

Add comment

d7b269b

mrodm added 6 commits October 21, 2025 10:22

Check success variable in import saved objects response

6542265

Apply same changes for edit command when running with Serverless

a8b59fb

Remove unused parameters and fields

b6dccd2

Re-phrase new dashboard option

a2b0823

Re-order function

53cdeb3

Rename struct to get response from bulk_assets

9cb72c3

mrodm commented Oct 21, 2025

View reviewed changes

mrodm added 2 commits October 21, 2025 16:45

Make private more structs

14db2d7

Delete comment

243539a

mrodm commented Oct 21, 2025

View reviewed changes

cmd/export_dashboards.go Outdated Show resolved Hide resolved

mrodm commented Oct 21, 2025

View reviewed changes

mrodm marked this pull request as ready for review October 21, 2025 16:19

mrodm requested a review from a team October 21, 2025 16:19

elastic-vault-github-plugin-prod bot mentioned this pull request Oct 22, 2025

Test elastic-package#3007 - DO NOT MERGE elastic/integrations#15710

Closed

jsoriano approved these changes Oct 22, 2025

View reviewed changes

mrodm and others added 2 commits October 22, 2025 11:31

Update question to refer to just one package

402339c

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

Rephrase option to find a new dashboard

1069e50

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

mrodm added 6 commits October 22, 2025 11:38

Set as last option to look for a new dashboard

7775a0a

Move logic from exportAllDashboards function to FindServerlessDashboards

f491e12

Rename function

1ab5ec3

Inline functions to promt dashbaords

0609cae

Use the new UX from stack versions 9.0.0 and Serverless

9d4417a

Fix imports

9d9c0e2

jsoriano approved these changes Oct 22, 2025

View reviewed changes

mrodm changed the title ~~Allow to export dashboards from Serverless environments~~ Allow to export and edit dashboards from Serverless environments Oct 23, 2025

mrodm merged commit b214464 into elastic:main Oct 23, 2025
3 checks passed

mrodm deleted the update_export_dashboards_apis branch October 23, 2025 08:12

mrodm mentioned this pull request Nov 6, 2025

Show errors if any when updating managed setting in saved object #3057

Merged

Conversation

mrodm commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Author's checklist

Example of usage

How to test this PR locally

Uh oh!

mrodm Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mrodm Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

jsoriano Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

mrodm commented Oct 21, 2025

Uh oh!

mrodm Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mrodm Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

mrodm commented Oct 22, 2025

Uh oh!

elastic-vault-github-plugin-prod bot commented Oct 22, 2025

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jsoriano Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Oct 22, 2025

💚 Build Succeeded

History

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mrodm commented Oct 20, 2025 •

edited

Loading

mrodm Oct 20, 2025 •

edited

Loading