Tag: api

Gitlab Application settings API

Get current application settings

List the current application settings of the GitLab instance.

GET /application/settings
curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/application/settings

Example response:

{
   "default_projects_limit" : 100000,
   "signup_enabled" : true,
   "id" : 1,
   "default_branch_protection" : 2,
   "restricted_visibility_levels" : [],
   "password_authentication_enabled_for_web" : true,
   "after_sign_out_path" : null,
   "max_attachment_size" : 10,
   "user_oauth_applications" : true,
   "updated_at" : "2016-01-04T15:44:55.176Z",
   "session_expire_delay" : 10080,
   "home_page_url" : null,
   "default_snippet_visibility" : "private",
   "outbound_local_requests_whitelist": [],
   "domain_whitelist" : [],
   "domain_blacklist_enabled" : false,
   "domain_blacklist" : [],
   "created_at" : "2016-01-04T15:44:55.176Z",
   "default_project_visibility" : "private",
   "default_group_visibility" : "private",
   "gravatar_enabled" : true,
   "sign_in_text" : null,
   "container_registry_token_expire_delay": 5,
   "repository_storages": ["default"],
   "plantuml_enabled": false,
   "plantuml_url": null,
   "terminal_max_session_time": 0,
   "polling_interval_multiplier": 1.0,
   "rsa_key_restriction": 0,
   "dsa_key_restriction": 0,
   "ecdsa_key_restriction": 0,
   "ed25519_key_restriction": 0,
   "first_day_of_week": 0,
   "enforce_terms": true,
   "terms": "Hello world!",
   "performance_bar_allowed_group_id": 42,
   "instance_statistics_visibility_private": false,
   "user_show_add_ssh_key_message": true,
   "local_markdown_version": 0,
   "allow_local_requests_from_hooks_and_services": true,
   "allow_local_requests_from_web_hooks_and_services": true,
   "allow_local_requests_from_system_hooks": false,
   "asset_proxy_enabled": true,
   "asset_proxy_url": "https://assets.example.com",
   "asset_proxy_whitelist": ["example.com", "*.example.com", "your-instance.com"]
}

Example of using Apache Bench (ab) to POST JSON to an API

# post_loc.txt contains the json you want to post
# -p means to POST it
# -H adds an Auth header (could be Basic or Token)
# -T sets the Content-Type
# -c is concurrent clients
# -n is the number of requests to run in the test

ab -p post_loc.txt -T application/json -H 'Authorization: Token abcd1234' -c 10 -n 2000 http://example.com/api/v1/locations/

 

How to trust self-signed certificate in cURL command line?

Problem:

I’ve created a self-signed certificate for foo.localhost using a Let’s Encrypt recommendation using this Makefile:

include ../.env

configuration = csr.cnf
certificate = self-signed.crt
key = self-signed.key

.PHONY: all
all: $(certificate)

$(certificate): $(configuration)
    openssl req -x509 -out $@ -keyout $(key) -newkey rsa:2048 -nodes -sha256 -subj '/CN=$(HOSTNAME)' -extensions EXT -config $(configuration)

$(configuration):
    printf "[dn]\nCN=$(HOSTNAME)\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:$(HOSTNAME)\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth" > $@

.PHONY: clean
clean:
    $(RM) $(configuration)

I’ve then assigned that to a web server. I’ve verified that the server returns the relevant certificate:

$ openssl s_client -showcerts -connect foo.localhost:8443 < /dev/null
CONNECTED(00000003)
depth=0 CN = foo.localhost
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = foo.localhost
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/CN=foo.localhost
   i:/CN=foo.localhost
-----BEGIN CERTIFICATE-----
[…]
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=foo.localhost
issuer=/CN=foo.localhost
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1330 bytes and written 269 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: […]
    Session-ID-ctx: 
    Master-Key: […]
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    […]

    Start Time: 1529622990
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
    Extended master secret: no
---
DONE

How do I make cURL trust it without modifying anything in /etc? --cacert does not work, presumably because there is no CA:

$ curl --cacert tls/foo.localhost.crt 'https://foo.localhost:8443/'
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

The goal is to enable HTTPS during development:

  • I can’t have a completely production-like certificate without a lot of work to enable DNS verification in all development environments. Therefore I have to use a self-signed certificate.
  • I still obviously want to make my development environment as similar as possible to production, so I can’t simply ignore any and all certificate issues. curl -k is like catch (Exception e) {}in this case – nothing at all like a browser talking to a web server.

In other words, when running curl [something] https://project.local/api/foo I want to be confident that

  1. if TLS is configured properly except for having a self-signed certificate the command will succeed and
  2. if I have any issues with my TLS configuration except for having a self-signed certificate the command will fail.

Using HTTP or --insecure fails the second criterion.

Solution:

Following these steps should solve your issue:

  1. Download and save the self-signed certificate: echo quit | openssl s_client -showcerts -servername "${API_HOST}" -connect "${API_HOST}":443 > cacert.pem
  2. Tell the curl client about it: curl --cacert cacert.pem --location --silent https://${API_HOST}

Also one could use wget and ignore certificates with: wget --no-check-certificate https://${API_HOST}

Wazuh: Issues encountered and solutions

 

  1. logstash service does not find config files in /etc/logstash/conf.d
    I installed logstash via centos rpm and placed a valid logstash configuration file into /etc/logstash/conf.d. Starting the service it fails with following error message:

    {:timestamp=>"2015-10-21T08:11:06.939000+0000", :message=>"Error: No config files found: /etc/logstash/conf.d/pipe.conf\nCan you make sure this path is a logstash config file?", :file=>"logstash/agent.rb", :line=>"159", :method=>"execute"}
{:timestamp=>"2015-10-21T08:11:06.948000+0000", :message=>"You may be interested in the '--configtest' flag which you can\nuse to validate logstash's configuration before you choose\nto restart a running system.", :file=>"logstash/agent.rb", :line=>"161", :method=>"execute"}

    there is definitely a file in this location read/write to anyone

    [root@logserv logstash]# ls -al /etc/logstash/conf.d/
total 12
drwxr-xr-x. 2 root root 4096 Oct 21 08:00 .
drwx------. 4 root root 4096 Oct 21 07:54 ..
-rwxrwxrwx. 1 root root  234 Oct 21 07:56 pipe.conf

    and its content is valid

    [root@logserv logstash]# bin/logstash -f /etc/logstash/conf.d/pipe.conf --configtest
Configuration OK

    Solution:

    chown -R logstash:root /etc/logstash/conf.d
chmod 0750 /etc/logstash/conf.d
chmod 0640 /etc/logstash/conf.d/*
  2.  If in the Wazuh UI you see data in wazuh-alerts but not in any of the wazuh dashboards, check if the data is getting pushed to Elasticsearch first:

    curl localhost:9200/_cat/indices

    The output should look like below:

    green open wazuh-alerts-3.x-2019.07.19 GIPOTyJuSxSZgVtsdkouxg 3 0 131 0 424.7kb 424.7kb
green open .kibana_task_manager cCFAzTqIQ6GuhVtJsfuUrQ 1 0 2 0 29.5kb 29.5kb
yellow open .wazuh tgqyhP1rQHqRk4bnfvjivg 1 1 1 0 11kb 11kb
green open wazuh-alerts-3.x-2019.07.20 vbSs-0TRRRKihI3vo67C0w 3 0 10 0 79.7kb 79.7kb
green open wazuh-alerts-3.x-2019.07.21 GYbynBOLTsedyuxIVfSmig 3 0 9 0 80.7kb 80.7kb
green open .kibana_1 24p2awqCTFafufPXuTkM_A 1 0 6 2 110.6kb 110.6kb
green open wazuh-monitoring-3.x-2019.07.18 GtPTclhVS6CveIoTB9s88w 2 0 192 0 174.7kb 174.7kb
green open wazuh-monitoring-3.x-2019.07.20 skX7aKIMTNa20VKvZdG-gg 2 0 192 0 210.9kb 210.9kb
green open wazuh-monitoring-3.x-2019.07.17 fERZ9LMeQheBUDo4CFZgbw 2 0 98 0 215.1kb 215.1kb
green open wazuh-monitoring-3.x-2019.07.21 fDT71M7bSNawPplieIEXRg 2 0 46 0 208.6kb 208.6kb
yellow open .wazuh-version 2TPSH17YQ4e_n6NiWDhQqQ 1 1 1 0 5.2kb 5.2kb
green open wazuh-monitoring-3.x-2019.07.19 8RDIhk0EQIOxNIYOOh6VXA 2 0 198 0 140.1kb 140.1kb

    Check if wazuh-alerts-3.x-* index is present with current date. If yes, then check if data is present in the index:

    curl localhost:9200/<INDEX_NAME>/_search?pretty=true&size=1000
    Example:
    localhost:9200/wazuh-alerts-3.x-2019.07.19/_search?pretty=true&size=1000

    This should return first 1000 entries.
    Check if latest entries are present. Also check if manager.name matches the hostname of the manager. If not, then change the hostname of the manager by executing command:

    hostname <HOSTNAME>
    Example:
    hostname abc.example.com

    The alerts in the Elasticsearch index will start coming in with the manager.hostname as abc.example.com

  3. Get details of a Wazuh agent:

    curl -u foo:bar localhost:55000/agents/<AGENT_ID>

    foo:bar is the credentials for wazuh api.

  4. Get details of nodes in Wazuh cluster: 
    curl -u foo:bar localhost:55000/cluster/nodes?pretty
  5. Rules not getting reflected aftere change in /var/ossec/ruleset/rules
    Restarting the wazuh-manger should reload the rules:

    systemctl restart wazuh-manager

  6. No events in wazuh-monitoring index:
    Check if index wazuh-monitoring-3.x-* is present with today’s date:

    curl elastic:9200/_cat/indices/wazuh-monitoring*

    Check if there is any error in wazuhapp for wazuh-monitoring:

    cat /usr/share/kibana/optimize/wazuh-logs/wazuhapp-plain.log | grep monitoring

    Execute:

    curl -XGET "http://elastic:9200/_cat/templates/wazuh-agent"

    If you get something like:
    wazuh-agent [wazuh-monitoring*, wazuh-monitoring-3.x-*] 0
    You probably have a template issue. Execute the following to resolve it:

    Stop Kibana: systemctl stop kibana

Delete the monitoring template, curl -XDELETE elastic:9200/_template/wazuh-agent

Restart Kibana: systemctl restart kibana, it should insert the monitoring template and the Kibana UI should start working shortly
  7. Change wazuh app to debug mode:
    Edit /usr/share/kibana/plugins/wazuh/config.yml
    Replace #logs.level: info with logs.level: debug, then restart Kibana service (systemctl restart kibana)
  8. Wazuh UI Error: “Saved field parameter is now invalid” OR “Error in Visualisation: field is a required parameter”
    You will require a cleanup. Execute the following commands:
    Delete wazuh-alerts-3.x-* index with today’s date:

    curl -XDELETE localhost:9200/wazuh-alerts-3.x-2019.07.18
systemctl restart wazuh-manager
curl -XDELETE localhost:9200/.kiban*
systemctl restart kibana
rm -f /var/ossec/queue/db/0*
systemctl restart wazuh-manager
  9. Generate SCA alerts: 
    rm -f /var/ossec/queue/db/0*
systemctl restart wazuh-manager
  10. Error in visualisation: Expected numeric type on field on data.sca.score got numeric:
    You most probably have wrong template. Just install the template according to your wazuh version from their github repo. To install latest (3.9.3) execute the following:

    curl https://raw.githubusercontent.com/wazuh/wazuh/v3.9.3/extensions/elasticsearch/6.x/wazuh-template.json | curl -X PUT "http://localhost:9200/_template/wazuh" -H 'Content-Type: application/json' -d @-
  11. java.lang.IllegalArgumentException: Rejecting mapping update to [wazuh-alerts-3.x] as the final mapping would have more than 1 type:
    This would most possibly mean you have wrong template. Install the latest one with above step.
    If already done you might have an issue with your logstash configuration:

    curl -so /etc/logstash/conf.d/01-wazuh.conf https://raw.githubusercontent.com/wazuh/wazuh/v3.9.3/extensions/logstash/7.x/01-wazuh-remote.conf

    Delete wazuh-alerts-3.x-* index with today’s date:

    curl -XDELETE localhost:9200/wazuh-alerts-3.x-2019.07.18systemctl restart wazuh-manager
curl -XDELETE localhost:9200/.kiban*
systemctl restart kibana
rm -f /var/ossec/queue/db/0*
systemctl restart wazuh-manager
  12. Version mismatch:
    If you get the above error on Wazuh UI, execute the following commands:

    service kibana stop
curl -XDELETE localhost:9200/.wazuh
curl -XDELETE localhost:9200/.wazuh_version
service kibana start
  13. Other useful commands: 
    Check documents in an index:
curl elastic:9200/_cat/indices/wazuh-monitoring*

Check wazuhapp logs:
cat /usr/share/kibana/optimize/wazuh-logs/wazuhapp-plain.log | grep monitoring

Check wazuh config:
cat /usr/share/kibana/plugins/wazuh/config.yml

Get Wazuh id:
curl elastic:9200/.wazuh/_search?pretty -s | grep "_id"

Templates:
curl elastic:9200/_cat/templates/wazuh

Version:
cat /usr/share/kibana/plugins/wazuh/package.json | grep version
cat /etc/ossec-init.conf | grep -i version
curl -u foo:bar localhost:55000/version

Monitoring settings:
cat /usr/share/kibana/plugins/wazuh/config.yml | grep monitoring

Search monitoring index:
curl elastic:9200/wazuh-monitoring*/_search

List of agents:
curl -u foo:bar "localhost:55000/agents?q=id!=000"

Index settings:
curl  "elastic:9200/wazuh-monitoring-3.x-2019.07.17/_settings"

Get details of template:
curl -XGET "http://elastic:9200/_cat/templates/wazuh-agent"

Check if filebeat is configured correctly:
filebeat test output

Install and configure Wazuh with ELK 6.x

Wazuh helps you to gain deeper security visibility into your infrastructure by monitoring hosts at an operating system and application level. This solution, based on lightweight multi-platform agents, provides the following capabilities:

  • File integrity monitoring

Wazuh monitors the file system, identifying changes in content, permissions, ownership, and attributes of files that you need to keep an eye on.

 

  • Intrusion and anomaly detection

Agents scan the system looking for malware, rootkits or suspicious anomalies. They can detect hidden files, cloaked processes or unregistered network listeners, as well as inconsistencies in system call responses.

 

  • Automated log analysis

Wazuh agents read operating system and application logs, and securely forward them to a central manager for rule-based analysis and storage. The Wazuh rules help make you aware of application or system errors, misconfigurations, attempted and/or successful malicious activities, policy violations and a variety of other security and operational issues.

 

  • Policy and compliance monitoring

Wazuh monitors configuration files to ensure they are compliant with your security policies, standards and/or hardening guides. Agents perform periodic scans to detect applications that are known to be vulnerable, unpatched, or insecurely configured.

This diverse set of capabilities is provided by integrating OSSEC, OpenSCAP and Elastic Stack into a unified solution and simplifying their configuration and management.

Execute the following commands to install and configure Wazuh:

  1. apt-get update

  2. apt-get install curl apt-transport-https lsb-release gnupg2

  3. curl -s https://packages.wazuh.com/key/GPG-KEY-WAZUH | apt-key add –

  4. echo “deb https://packages.wazuh.com/3.x/apt/ stable main” | tee -a /etc/apt/sources.list.d/wazuh.list

  5. apt-get update

  6. apt-get install wazuh-manager

  7. systemctl status wazuh-manager

  8. curl -sL https://deb.nodesource.com/setup_8.x | bash –

  9. apt-get install gcc g++ make

  10. apt-get install -y nodejs

  11. curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add –

  12. echo “deb https://dl.yarnpkg.com/debian/ stable main” | sudo tee /etc/apt/sources.list.d/yarn.list

  13. sudo apt-get update && sudo apt-get install yarn

  14. apt-get install nodejs

  15. apt-get install wazuh-api

  16. systemctl status wazuh-api

  17. sed -i “s/^deb/#deb/” /etc/apt/sources.list.d/wazuh.list

  18. apt-get update

  19. curl -s https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add –

  20. echo “deb https://artifacts.elastic.co/packages/6.x/apt stable main” | tee /etc/apt/sources.list.d/elastic-6.x.list

  21. apt-get update

  22. apt-get install filebeat

  23. curl -so /etc/filebeat/filebeat.yml https://raw.githubusercontent.com/wazuh/wazuh/v3.9.3/extensions/filebeat/6.x/filebeat.yml

  24. Edit the file /etc/filebeat/filebeat.yml and replace  YOUR_ELASTIC_SERVER_IP with the IP address or the hostname of the Logstash server.

  25. apt search elasticsearch

  26. apt-get install elasticsearch

  27. systemctl daemon-reload

  28. systemctl enable elasticsearch.service

  29. systemctl start elasticsearch.service

  30. curl https://raw.githubusercontent.com/wazuh/wazuh/v3.9.3/extensions/elasticsearch/6.x/wazuh-template.json | curl -X PUT “http://localhost:9200/_template/wazuh&#8221; -H ‘Content-Type: application/json’ -d @-

  31. curl -X PUT “http://localhost:9200/*/_settings?pretty&#8221; -H ‘Content-Type: application/json’ -d’
    “settings”: {
    “number_of_replicas” : 0
    }
    ‘

  32. sed -i ‘s/#bootstrap.memory_lock: true/bootstrap.memory_lock: true/’ /etc/elasticsearch/elasticsearch.yml

  33. sed -i ‘s/^-Xms.*/-Xms12g/;s/^-Xmx.*/-Xmx12g/’ /etc/elasticsearch/jvm.options

  34. mkdir -p /etc/systemd/system/elasticsearch.service.d/

  35. echo -e “[Service]\nLimitMEMLOCK=infinity” > /etc/systemd/system/elasticsearch.service.d/elasticsearch.conf

  36. systemctl daemon-reload

  37. systemctl restart elasticsearch

  38. apt-get install logstash

  39. curl -so /etc/logstash/conf.d/01-wazuh.conf https://raw.githubusercontent.com/wazuh/wazuh/master/extensions/logstash/6.x/01-wazuh-local.conf

  40. systemctl daemon-reload

  41. systemctl enable logstash.service

  42. systemctl start logstash.service

  43. systemctl status filebeat

  44. systemctl start filebeat

  45. apt-get install kibana

  46. export NODE_OPTIONS=”–max-old-space-size=3072″

  47. sudo -u kibana /usr/share/kibana/bin/kibana-plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-3.9.3_6.8.1.zip

  48. Kibana will only listen on the loopback interface (localhost) by default. To set up Kibana to listen on all interfaces, edit the file /etc/kibana/kibana.yml uncommenting the setting server.host. Change the value to:
    server.host: “0.0.0.0”

  49. systemctl enable kibana.service

  50. systemctl start kibana.service

  51. cd /var/ossec/api/configuration/auth

  52. Create a username and password for Wazuh API. When prompted, enter the password:
    node htpasswd -c user admin

  53. systemctl restart wazuh-api

Then in the agent machine execute the following commands:

  1. apt-get install curl apt-transport-https lsb-release gnupg2
  2. curl -s https://packages.wazuh.com/key/GPG-KEY-WAZUH | apt-key add –
  3. echo “deb https://packages.wazuh.com/3.x/apt/ stable main” | tee /etc/apt/sources.list.d/wazuh.list
  4. apt-get update
  5. You can automate the agent registration and configuration using variables. It is necessary to define at least the variable WAZUH_MANAGER_IP. The agent will use this value to register and it will be the assigned manager for forwarding events.
    WAZUH_MANAGER_IP=“10.0.0.2” apt-get install wazuh-agent
  6. sed -i “s/^deb/#deb/” /etc/apt/sources.list.d/wazuh.list
  7. apt-get update

In this section, we’ll register the Wazuh API (installed on the Wazuh server) into the Wazuh App in Kibana:

  1. Open a web browser and go to the Elastic Stack server’s IP address on port 5601 (default Kibana port). Then, from the left menu, go to the Wazuh App.

    ../../_images/kibana_app.png

  1. Click on Add new API.
    ../../_images/connect_api.png


  2. Fill Username and Password with appropriate credentials you created in previous step. Enter http://MANAGER_IP for the URL, where MANAGER_IP is the real IP address of the Wazuh qserver. Enter “55000” for the Port.

    ../../_images/fields_api.png

  3. Click on Save.

    ../../_images/app_running.png

docker registry api summary

A list of methods and URIs are covered in the table below:

Method Path Entity Description
GET /v2/ Base Check that the endpoint implements Docker Registry API V2.
GET /v2/<name>/tags/list Tags Fetch the tags under the repository identified by name.
GET /v2/<name>/manifests/<reference> Manifest Fetch the manifest identified by name and reference where reference can be a tag or digest. A HEAD request can also be issued to this endpoint to obtain resource information without receiving all data.
PUT /v2/<name>/manifests/<reference> Manifest Put the manifest identified by name and reference where reference can be a tag or digest.
DELETE /v2/<name>/manifests/<reference> Manifest Delete the manifest identified by name and reference. Note that a manifest can only be deleted by digest.
GET /v2/<name>/blobs/<digest> Blob Retrieve the blob from the registry identified by digest. A HEAD request can also be issued to this endpoint to obtain resource information without receiving all data.
DELETE /v2/<name>/blobs/<digest> Blob Delete the blob identified by name and digest
POST /v2/<name>/blobs/uploads/ Initiate Blob Upload Initiate a resumable blob upload. If successful, an upload location will be provided to complete the upload. Optionally, if the digest parameter is present, the request body will be used to complete the upload in a single request.
GET /v2/<name>/blobs/uploads/<uuid> Blob Upload Retrieve status of upload identified by uuid. The primary purpose of this endpoint is to resolve the current status of a resumable upload.
PATCH /v2/<name>/blobs/uploads/<uuid> Blob Upload Upload a chunk of data for the specified upload.
PUT /v2/<name>/blobs/uploads/<uuid> Blob Upload Complete the upload specified by uuid, optionally appending the body as the final chunk.
DELETE /v2/<name>/blobs/uploads/<uuid> Blob Upload Cancel outstanding upload processes, releasing associated resources. If this is not called, the unfinished uploads will eventually timeout.
GET /v2/_catalog Catalog Retrieve a sorted, json list of repositories available in the registry.

PAGINATION

Paginated catalog results can be retrieved by adding an n parameter to the request URL, declaring that the response should be limited to n results. Starting a paginated flow begins as follows:

GET /v2/_catalog?n=<integer>

The above specifies that a catalog response should be returned, from the start of the result set, ordered lexically, limiting the number of results to n. The response to such a request would look as follows:

200 OK
Content-Type: application/json
Link: <<url>?n=<n from the request>&last=<last repository in response>>; rel="next"

{
  "repositories": [
    <name>,
    ...
  ]
}

The above includes the first n entries from the result set. To get the next n entries, one can create a URL where the argument last has the value from repositories[len(repositories)-1]. If there are indeed more results, the URL for the next block is encoded in an RFC5988 Link header, as a “next” relation. The presence of the Link header communicates to the client that the entire result set has not been returned and another request must be issued. If the header is not present, the client can assume that all results have been received.

NOTE: In the request template above, note that the brackets are required. For example, if the url ishttp://example.com/v2/_catalog?n=20&last=b, the value of the header would be <http://example.com/v2/_catalog?n=20&last=b>; rel="next". Please see RFC5988 for details.

Compliant client implementations should always use the Link header value when proceeding through results linearly. The client may construct URLs to skip forward in the catalog.

To get the next result set, a client would issue the request as follows, using the URL encoded in the described Link header:

GET /v2/_catalog?n=<n from the request>&last=<last repository value from previous response>

The above process should then be repeated until the Link header is no longer set.

The catalog result set is represented abstractly as a lexically sorted list, where the position in that list can be specified by the query term last. The entries in the response start after the term specified by last, up to n entries.

The behavior of last is quite simple when demonstrated with an example. Let us say the registry has the following repositories:

a
b
c
d

If the value of n is 2, a and b will be returned on the first response. The Link header returned on the response will have n set to 2 and last set to b:

Link: <<url>?n=2&last=b>; rel="next"

The client can then issue the request with the above value from the Link header, receiving the values c and d. Note that n may change on the second to last response or be fully omitted, depending on the server implementation.

 

 

Modify a virtual machine image with guestfish

Guestfish is a shell and command-line tool for examining and modifying virtual machine filesystems. It uses libguestfs and exposes all of the functionality of the guestfs API.

Sometimes you must modify a virtual machine image to remove any traces of the MAC address that was assigned to the virtual network interface card when the image was first created. This is because the MAC address is different when the virtual machine images boots. This example shows how to use the guestfish to remove references to the old MAC address by deleting the/etc/udev/rules.d/70-persistent-net.rules file and removing the HWADDR line from the /etc/sysconfig/network-scripts/ifcfg-eth0 file.

Assume that you have a CentOS qcow2 image called centos63_desktop.img. Mount the image in read-write mode as root, as follows:

# guestfish --rw -a centos.img

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help on commands
'man' to read the manual
'quit' to quit the shell

><fs>

This starts a guestfish session.

Notethe guestfish prompt looks like a fish: ><fs>.

We must first use the run command at the guestfish prompt before we can do anything else. This will launch a virtual machine, which will be used to perform all of the file manipulations.

><fs> run
  1. We can now view the file systems in the image using the list-filesystems command: 
    ><fs> list-filesystems
/dev/vda1: ext4
/dev/vg_centosbase/lv_root: ext4
/dev/vg_centosbase/lv_swap: swap
  2. We need to mount the logical volume that contains the root partition: 
    ><fs> mount /dev/vg_centosbase/lv_root /
  3. Next, we want to delete a file. We can use the rm guestfish command, which works the same way it does in a traditional shell. 
    ><fs> rm /etc/udev/rules.d/70-persistent-net.rules
  4. We want to edit the ifcfg-eth0 file to remove the HWADDR line. The edit command will copy the file to the host, invoke your editor, and then copy the file back. 
    ><fs> edit /etc/sysconfig/network-scripts/ifcfg-eth0
  5. If you want to modify this image to load the 8021q kernel at boot time, you must create an executable script in the/etc/sysconfig/modules/ directory. You can use the touch guestfish command to create an empty file, the edit command to edit it, and the chmod command to make it executable. 
    ><fs> touch /etc/sysconfig/modules/8021q.modules
><fs> edit /etc/sysconfig/modules/8021q.modules
  6. We add the following line to the file and save it: 
    modprobe 8021q
  7. Then we set to executable: 
    ><fs> chmod 0755 /etc/sysconfig/modules/8021q.modules
  8. We are done, so we can exit using the exit command: 
    ><fs> exit

Example 2: Add passwordless sudo to a user:

guestfish --rw -a centos.img
><fs> run
><fs> list-filesystems
/dev/sda1: ext4
><fs> mount /dev/sda1 /
><fs> touch /etc/sudoers.d/user1-sudo
><fs> vi /etc/sudoers.d/useer1-sudo
><fs> quit

This will add the sudoers file and enable passwordless sudo for user1

List all the images in docker registry

The command to list all the images in registry is:

curl http://<IP/Hostname>:<Port>/v2/_catalog | python -mjson.tool

See the versions for a specific image (example: aerospike):

curl http://<IP/Hostname>:<Port>/v2/aerospike/tags/list | python -mjson.tool

how to detect if api calls are fraud

I would like to let people vote on something inside an iframe-widget, which is embedded from a different domain. I don’t have access to parent-(embedding) page.
(like “rate this product from 1-5 stars”)

they should not(!)..

  • ..need to login
  • ..need to prove “iam a human” with some widget
  • ..be able to vote multiple times
  • ..easily be able to to easily (I know I can’t prevent 100%) manipulate the voting system using some easy way. Like opening web-inspector and copying the request in https://www.getpostman.com

the solution..

  • ..should be as simple as possible (so maybe not deep learning or such things)
  • ..does not need to be 100% secure
  • ..should be able to be done using Amazon-Web-Service products

..i do have some ideas involving minified javascript functions and checking the result server side but I would like to keep the question unbiased to see which ideas come up..

thank you in advance!

Solution:

You can integrate a bitcoin proof of work captcha, and make money out of people voting on you iframe widget. Some one might vote a million times, but you will make money! See https://coinhive.com/ for more info