Skip to content
Demis Bellot edited this page Oct 25, 2016 · 10 revisions

This page has moved to docs.servicestack.net/csv-format


The CSV format is a first-class supported format which means all your existing web services can automatically take accept and return CSV without any config or code changes.

Importance of CSV

CSV is an important format for transferring, migrating and quickly visualizing data as all spreadsheets support viewing and editing CSV files directly whilst its supported by most RDBMS support exporting and importing data. Compared with other serialization formats, it provides a compact and efficient way to transfer large datasets in an easy to read text format.

Speed

The CSV Serializer used was developed using the same tech that makes ServiceStack's JSV and JSON serializers fast (i.e. no run-time reflection, static delegate caching, etc), which should make it the fastest CSV serializer available for .NET.

Downloadable Separately

The CsvSerializer is maintained in the ServiceStack.Text project which can be downloaded from NuGet at:

PM> Install-Package ServiceStack.Text

How to register your own custom format with ServiceStack

What makes the 'CSV' format different is its the first format added using the new extensions API. The complete code to register the CSV format is:

//Register the 'text/csv' content-type and serializers 
//(format is inferred from the last part of the content-type)

this.ContentTypes.Register(ContentType.Csv,
    CsvSerializer.SerializeToStream, CsvSerializer.DeserializeFromStream);

//ResponseFilter to add 'Content-Disposition' header for browsers to open in Spreadsheet
this.ResponseFilters.Add((req, res, dto) =>
{
    if (req.ResponseContentType == ContentType.Csv)
    {
        res.AddHeader(HttpHeaders.ContentDisposition,
            string.Format("attachment;filename={0}.csv", req.OperationName));
    }
});

Note: ServiceStack already does this for you though it still serves a good example to show how you can plug-in your own custom format. If you wish, you can remove all custom formats with (inside AppHost.Configure()):

this.ContentTypes.ClearCustomFilters();

The ability to automatically to register another format and provide immediate value and added functionality to all your existing web services (without any code-changes or configuration) we believe is a testament to ServiceStack's clean design of using strongly-typed 'message-based' DTOs to let you develop clean, testable and re-usable web services. No code-gen or marshalling is required to bind to an abstract method signature, every request and calling convention maps naturally to your webservices DTOs.

Usage

The CSV format is effectively a first-class supported format so everything should work as expected, including being registered as an available format on ServiceStack's metadata index page:

And being able to preview the output of a service:

By default they are automatically available using ServiceStack's standard calling conventions, e.g:

REST Usage

CSV also works just as you would expect with user-defined REST-ful urls, i.e. you can append ?format=csv to specify the format in the url e.g:

This is how the above web service output looks when opened up in google docs

Alternative in following with the HTTP specification you can also specify content-type "text/csv" in the Accept header of your HttpClient as done in HTTP Utils extension methods:

var csv = "http://nortwind.servicestack.net/customers".GetCsvFromUrl();

CSV Deserialization Support

The introduction of the new AutoQuery Data feature and it's MemorySource has made full CSV support a lot more appealing which caused CSV Deserialization support where it's implementation is now complete. This now unlocks the ability to create fully-queryable Services over flat-file .csv's (or Excel spreadsheets exported to .csv) by just deserializing CSV into a List of POCO's and registering it with AutoQuery Data:

var pocos = File.ReadAllText("path/to/data.csv").FromCsv<List<Poco>>();

//AutoQuery Data Plugin
Plugins.Add(new AutoQueryDataFeature()
    .AddDataSource(ctx => ctx.MemorySource(pocos)));

// AutoQuery DTO
[Route("/pocos")]
public class QueryPocos : QueryData<Poco> {}

Super CSV Format

A noteworthy feature that sets ServiceStack's CSV support apart is that it's built on the compact and very fast JSV Format which not only can deserialize a tabular flat file of scalar values at high-speed, it also supports deeply nested object graphs which are encoded in JSV and escaped in a CSV field as normal. An example of this can be seen in a HTTP sample log fragment below where the HTTP Request Headers are a serialized from a Dictionary<string,string>:

Id,HttpMethod,AbsoluteUri,Headers
1,GET,http://localhost:55799,"{Connection:keep-alive,Accept:""text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"",Accept-Encoding:""gzip, deflate, sdch"",Accept-Language:""en-US,en;q=0.8"",Host:""localhost:55799"",User-Agent:""Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36"",Upgrade-Insecure-Requests:1}"

Being such a versatile file format opens up a lot of new possibilities, e.g. instead of capturing seed data in code you could maintain them in plain-text .csv files and effortlessly load them on App Startup, e.g:

using (var db = container.Resolve<IDbConnectionFactory>().Open())
{
    if (db.CreateTableIfNotExists<Country>()) //returns true if Table created
    {
        List<Country> countries = "~/App_Data/countries.csv".MapHostAbsolutePath()
            .ReadAllText().FromCsv<List<Country>>();
    
        db.InsertAll(countries);
    }
}

All Services now accept CSV Content-Types

Another immediate benefit of CSV Deserialization is that now all Services can now process the CSV Content-Type. Being a tabular data format, CSV shines when it's processing a list of DTO's, one way to do that in ServiceStack is to have your Request DTO inherit List<T>:

[Route("/pocos")]
public class Pocos : List<Poco>, IReturn<Pocos>
{
    public Pocos() {}
    public Pocos(IEnumerable<Poco> collection) : base(collection) {}
}

It also behaves the same way as CSV Serialization but in reverse where if your Request DTO is annotated with either [DataContract] or the more explicit [Csv(CsvBehavior.FirstEnumerable)] it will automatically deserialize the CSV into the first IEnumerable property, so these 2 Request DTO's are equivalent to above:

[Route("/pocos")]
[DataContract]
public class Pocos : IReturn<Pocos>
{
    [DataMember]
    public List<Poco> Items { get; set; }
}

[Route("/pocos")]
[Csv(CsvBehavior.FirstEnumerable)]
public class Pocos : IReturn<Pocos>
{
    public List<Poco> Items { get; set; }
}

In addition to the above flexible options for defining CSV-friendly Services, there's also a few different options for sending CSV Requests to the above Services. You can use the new CSV PostCsvToUrl() extension methods added to HTTP Utils:

string csvText = File.ReadAllText("pocos.csv");

//Send CSV Text
List<Poco> response = "http://example.org/pocos"
    .PostCsvToUrl(csvText)
    .FromCsv<List<Poco>>();
    
//Send POCO DTO's
List<Poco> dtos = csvText.FromCsv<List<Poco>>();
List<Poco> response = "http://example.org/pocos"
    .PostCsvToUrl(dtos)
    .FromCsv<List<Poco>>();    

Alternatively you can use the CsvServiceClient which has the nice Typed API's you'd expect from a Service Client:

var client = new CsvServiceClient(baseUrl);

Pocos response = client.Post(new Pocos(dtos));

Ideal for Auto Batched Requests

The CsvServiceClient by virtue of being configured to use a well-defined Tabular data format is perfect for sending Auto-Batched Requests which by definition send a batch of POCO's making the CSV format the most compact text format to send them with:

var requests = new[]
{
    new Request { ... },
    new Request { ... },
    new Request { ... },
};

var responses = client.SendAll(requests);

Limitations

As most readers familiar with the CSV format will know there are some inherent limitations with CSV-format namely it is a flat-structured tabular data format that really only supports serialization of a single resultset.

This limitation remains, although if you decorate your Response DTO with a [Csv(CsvBehavior.FirstEnumerable)] or standard .NET [DataContract]/[DataMember] attributes the CSV Serializer will change to use the following conventions:

  • If you only return one result in your DTO it will serialize that.
  • If you return multiple results it will pick the first IEnumerable<> property or if it doesn't exist picks the first property.
  • Non-enumerable results are treated like a single row.

Basically if you only return 1 result it should work as expected otherwise it will chose the best candidate based on the rules above.

The second major limitation is that it doesn't yet include a CSV Deserializer (currently on the TODO list), so while you can view the results in CSV format you can't post data to your web service in CSV and have it automatically deserialize for you. You can however still upload a CSV file and parse it manually yourself.

Features

Unlike most CSV serializers that can only serialize rows of primitive values, the CsvSerializer uses the JSV Format under the hood so even complex types will be serialized in fields in a easy to read format - no matter how deep its hierarchy.



  1. Getting Started

    1. Creating your first project
    2. Create Service from scratch
    3. Your first webservice explained
    4. Example Projects Overview
    5. Learning Resources
  2. Designing APIs

    1. ServiceStack API Design
    2. Designing a REST-ful service with ServiceStack
    3. Simple Customer REST Example
    4. How to design a Message-Based API
    5. Software complexity and role of DTOs
  3. Reference

    1. Order of Operations
    2. The IoC container
    3. Configuration and AppSettings
    4. Metadata page
    5. Rest, SOAP & default endpoints
    6. SOAP support
    7. Routing
    8. Service return types
    9. Customize HTTP Responses
    10. Customize JSON Responses
    11. Plugins
    12. Validation
    13. Error Handling
    14. Security
    15. Debugging
    16. JavaScript Client Library (ss-utils.js)
  4. Clients

    1. Overview
    2. C#/.NET client
      1. .NET Core Clients
    3. Add ServiceStack Reference
      1. C# Add Reference
      2. F# Add Reference
      3. VB.NET Add Reference
      4. Swift Add Reference
      5. Java Add Reference
    4. Silverlight client
    5. JavaScript client
      1. Add TypeScript Reference
    6. Dart Client
    7. MQ Clients
  5. Formats

    1. Overview
    2. JSON/JSV and XML
    3. HTML5 Report Format
    4. CSV Format
    5. MessagePack Format
    6. ProtoBuf Format
  6. View Engines 4. Razor & Markdown Razor

    1. Markdown Razor
  7. Hosts

    1. IIS
    2. Self-hosting
    3. Messaging
    4. Mono
  8. Security

    1. Authentication
    2. Sessions
    3. Restricting Services
    4. Encrypted Messaging
  9. Advanced

    1. Configuration options
    2. Access HTTP specific features in services
    3. Logging
    4. Serialization/deserialization
    5. Request/response filters
    6. Filter attributes
    7. Concurrency Model
    8. Built-in profiling
    9. Form Hijacking Prevention
    10. Auto-Mapping
    11. HTTP Utils
    12. Dump Utils
    13. Virtual File System
    14. Config API
    15. Physical Project Structure
    16. Modularizing Services
    17. MVC Integration
    18. ServiceStack Integration
    19. Embedded Native Desktop Apps
    20. Auto Batched Requests
    21. Versioning
    22. Multitenancy
  10. Caching

  11. Caching Providers

  12. HTTP Caching 1. CacheResponse Attribute 2. Cache Aware Clients

  13. Auto Query

  14. Overview

  15. Why Not OData

  16. AutoQuery RDBMS

  17. AutoQuery Data 1. AutoQuery Memory 2. AutoQuery Service 3. AutoQuery DynamoDB

  18. Server Events

    1. Overview
    2. JavaScript Client
    3. C# Server Events Client
    4. Redis Server Events
  19. Service Gateway

    1. Overview
    2. Service Discovery
  20. Encrypted Messaging

    1. Overview
    2. Encrypted Client
  21. Plugins

    1. Auto Query
    2. Server Sent Events
    3. Swagger API
    4. Postman
    5. Request logger
    6. Sitemaps
    7. Cancellable Requests
    8. CorsFeature
  22. Tests

    1. Testing
    2. HowTo write unit/integration tests
  23. ServiceStackVS

    1. Install ServiceStackVS
    2. Add ServiceStack Reference
    3. TypeScript React Template
    4. React, Redux Chat App
    5. AngularJS App Template
    6. React Desktop Apps
  24. Other Languages

    1. FSharp
      1. Add ServiceStack Reference
    2. VB.NET
      1. Add ServiceStack Reference
    3. Swift
    4. Swift Add Reference
    5. Java
      1. Add ServiceStack Reference
      2. Android Studio & IntelliJ
      3. Eclipse
  25. Amazon Web Services

  26. ServiceStack.Aws

  27. PocoDynamo

  28. AWS Live Demos

  29. Getting Started with AWS

  30. Deployment

    1. Deploy Multiple Sites to single AWS Instance
      1. Simple Deployments to AWS with WebDeploy
    2. Advanced Deployments with OctopusDeploy
  31. Install 3rd Party Products

    1. Redis on Windows
    2. RabbitMQ on Windows
  32. Use Cases

    1. Single Page Apps
    2. HTML, CSS and JS Minifiers
    3. Azure
    4. Connecting to Azure Redis via SSL
    5. Logging
    6. Bundling and Minification
    7. NHibernate
  33. Performance

    1. Real world performance
  34. Other Products

    1. ServiceStack.Redis
    2. ServiceStack.OrmLite
    3. ServiceStack.Text
  35. Future

    1. Roadmap

Clone this wiki locally