> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gcore.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Raw Logs export

> Raw Logs exports CDN request logs hourly to S3, FTP, or SFTP storage with statuses, intervals, and log fields.

Raw Logs is the account-level hourly export of CDN request logs to Amazon Simple Storage Service (S3), File Transfer Protocol (FTP), or SSH File Transfer Protocol (SFTP) storage. Logs include requests to edge cache servers. Pre-cache server requests are included only when [Origin Shielding](/cdn/cdn-resource-options/general/enable-and-configure-origin-shielding) is enabled and **Add logs from Origin Shielding** is selected.

Raw Logs remains available for existing hourly S3, FTP, or SFTP exports. For new log-export setups, use [Logs Uploader](/cdn/logs/logs-uploader). Logs Uploader is the current workflow: it supports 5–60-minute delivery, selectable fields, HTTP(S) and Alibaba Cloud Simple Log Service (SLS) destinations, and per-resource policies. To move an existing Raw Logs configuration to Logs Uploader, contact [Gcore Support](mailto:support@gcore.com).

![Raw Logs page in the Customer Portal](https://mintcdn.com/gcore/lClv291c2f00rcjk/images/docs/cdn/logs/raw-logs-export-cdn-resource-logs-to-your-storage/raw-logs-page-off.png?fit=max\&auto=format\&n=lClv291c2f00rcjk\&q=85\&s=873da67cb012ef61f39427c1494bf0cc)

> [!NOTE]
>
> This is a paid feature. Contact [Gcore Support](mailto:support@gcore.com) to activate Raw Logs. After activation, enable **Receive Raw Logs** in the [Gcore Customer Portal](https://portal.gcore.com) and configure export to S3, FTP, or SFTP storage.

## Raw Logs settings

The **CDN** > **Logs** > **Raw Logs** page stores the export destination, empty-file preference, and Origin Shielding inclusion. Status, the hourly interval, and traffic totals come from that configuration.

### Origin Shielding log inclusion

When Origin Shielding is enabled for the account, the **Add logs from Origin Shielding** checkbox appears in Raw Logs settings. Select it so the export includes requests to both edge servers and the pre-cache server.

![Add logs from Origin Shielding checkbox](https://mintcdn.com/gcore/lClv291c2f00rcjk/images/docs/cdn/logs/raw-logs-export-cdn-resource-logs-to-your-storage/origin-shielding-logs.png?fit=max\&auto=format\&n=lClv291c2f00rcjk\&q=85\&s=b852b15e6aa22e58e3e35b43991f1ce3)

> [!TIP]
>
> If **Add logs from Origin Shielding** is not visible, Origin Shielding is not activated for the account. See the [Origin Shielding](/cdn/cdn-resource-options/general/enable-and-configure-origin-shielding) guide.

### Raw Logs statuses

The Raw Logs status in the Customer Portal can be:

* **Pending**: interval between connecting to storage and the first export
* **OK**: logs are exported from at least one CDN server
* **Failed**: a storage connection error occurred, or no export completed within 24 hours
* **Pause**: Raw Logs is paused

### Export interval

Logs are exported at the end of each hour. If Raw Logs is activated at 00:30, the first export runs between 00:45 and 01:00, and the next between 01:45 and 02:00.

If the covered CDN resources receive no requests during the export interval and **Do not send empty logs** is not selected, an empty log file (about 20 bytes) is sent to storage.

### Log traffic calculation

Raw Logs traffic totals depend on whether **Add logs from Origin Shielding** is selected. For CDN report metrics, see [View statistics on CDN resources](/cdn/view-statistics-of-a-cdn-resource).

> **Origin Shielding disabled**
>
> ```
> total_bytes = upstream_bytes + sent_bytes
> ```
>
> * `upstream_bytes` is `$upstream_response_length`: response length from the origin, in bytes
> * `sent_bytes` is `$bytes_sent`: bytes sent to the client from edge cache servers
>
> Example: `$upstream_response_length` `10485760` and `$bytes_sent` `1514848` produce `12000608` bytes.

> **Origin Shielding enabled**
>
> ```
> total_bytes = upstream_bytes + sent_bytes + shield_bytes
> ```
>
> * `upstream_bytes` is `$upstream_response_length`: response length from the origin, in bytes
> * `sent_bytes` is `$bytes_sent`: bytes sent to the client from edge cache servers
> * `shield_bytes` is `$bytes_sent` on the pre-cache server: bytes sent from the shielding node

> [!NOTE]
>
> Log totals can differ from billing statistics when some files are missing. Common causes:
>
> * Origin Shielding is enabled, but **Add logs from Origin Shielding** is not selected
> * The storage imposes a rate limit, so some files are not delivered to storage

### Log path example

```
s3://log-bucket-name/2019/08/20/15/nodename_cdn.example.com_access.log.gz
```

### Log example

```
"203.0.113.45" "-" "-" "[26/Apr/2019:09:47:40 +0000]" "GET /ContentCommon/images/image.png HTTP/1.1"
"200" "1514283" "https://example.com/videos/10" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 YaBrowser/16.10.0.2309 Safari/537.36"
"1514848" "[dh-up-gc18]" "https" "origin.example.com" "1.500" "0.445" "157" "bytes=0-1901653" "[dh]"
"MISS" "10485760" "192.0.2.1:80" "2510" "7399" "-" "-" "KZ" "-" "shield_no" "198.51.100.1" "80" "206" "-" "0.000"
"0.200" "198.51.100.2" "asnumber" "106980391" "1" "00-d5fe1dc9035165ce36952daf29686b6c-14330be33197dd1a-01" "-" "-"
"ECDHE-RSA-AES256-GCM-SHA384" "28a4184139cb43cdc79006cf2d1a4ac93bdc****" "r"
"application/json" "21" "PL" "45" "10" "100700"
```

### Log format

```
"$remote_addr" "-" "$remote_user" "[$time_local]" "$request" "$status"
"$body_bytes_sent" "$http_referer" "$http_user_agent" "$bytes_sent"
"$edgename" "$scheme" "$host" "$request_time"
"$upstream_response_time" "$request_length" "$http_range" "[$responding_node]"
"$upstream_cache_status" "$upstream_response_length" "$upstream_addr"
"$gcdn_api_client_id" "$gcdn_api_resource_id" "$uid_got" "$uid_set"
"$geoip_country_code" "$geoip_city" "$shield_type" "$server_addr" "$server_port"
"$upstream_status" "-" "$upstream_connect_time" "$upstream_header_time"
"$shard_addr" "$geoip2_data_asnumber" "$connection" "$connection_requests"
"$http_traceparent" "$http_x_forwarded_proto" "$gcdn_internal_status_code" "$ssl_cipher"
"$ssl_session_id" "$ssl_session_reused" "$sent_http_content_type" "$tcpinfo_rtt"
"$server_country_code" "$gcdn_tcpinfo_snd_cwnd" "$gcdn_tcpinfo_total_retrans" "$gcdn_rule_id"
"$request_completion" "$sent_http_content_length" "$gcdn_finalize_reason"
```

New fields can appear at the end of the line. When fields are added, Gcore sends an email notification.

### Log fields

The following table lists the available log fields. Fields marked as internal system variables belong to the CDN control plane and can be ignored for most analytics.

> **Log fields**
>
> | **Field** | **Log value example** | **Description** |
> | - | - | - |
> | `$remote_addr` | `203.0.113.45` | Client IP address |
> | `$remote_user` (internal system variable) | `-` | Username used in Basic authentication |
> | `[$time_local]` | `[26/Apr/2019:09:47:40 +0000]` | Local time in Common Log Format |
> | `$request` | `GET /ContentCommon/images/image.png HTTP/1.1` | HTTP method, requested file path, and HTTP version |
> | `$status` | `200` | Response status code from a CDN server |
> | `$body_bytes_sent` | `1514283` | Number of bytes sent to a user, excluding the response header size |
> | `$http_referer` | `https://example.com/videos/10` | Value of the `Referer` request header. The header usually identifies the page that initiated the request. |
> | `$http_user_agent` | `Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 YaBrowser/16.10.0.2309 Safari/537.36` | User agent that was used to send a request (browser or other application) |
> | `$bytes_sent` | `1514848` | Number of bytes sent to a user |
> | `$edgename` | `[dh-up-gc18]` | CDN server that forwarded a requested file |
> | `$scheme` | `https` | Protocol (HTTP or HTTPS) of a request |
> | `$host` | `cdn.example.com` | Requested hostname of a CDN resource |
> | `$request_time` | `1.500` | Request processing time in seconds (accurate to milliseconds); time elapsed between the first bytes of a request being processed and logging after the last bytes were sent to a user |
> | `$upstream_response_time` | `0.445` | Number of seconds (accurate to milliseconds) it took to receive a response from the upstream server. When [Origin Shielding](/cdn/cdn-resource-options/general/enable-and-configure-origin-shielding) is active, the upstream server may be an intermediate CDN cache rather than the origin. In case of multiple responses, commas and colons are used |
> | `$request_length` | `157` | Request length (including request line, header, and request body) |
> | `$http_range` | `bytes=0-1901653` | File fragment size in a Range request |
> | `[$responding_node]` | `dh` | Responding data center |
> | `$upstream_cache_status` | `MISS` | Status of a requested file in CDN cache: `HIT`: response served from the CDN cache. `STALE`: outdated response that failed to update (origin not responding or responding incorrectly). `UPDATING`: outdated response still updating from a previous request. `REVALIDATED`: response matching one on an origin (based on the `proxy_cache_revalidate` directive). `EXPIRED`: response that has expired in cache but still matches one on an origin; a request was sent to re-cache it. `MISS`: response served from the origin rather than from cache, including the first request after a purge. `BYPASS`: response served from the upstream server because configured cache-bypass conditions were met — for example a matching cookie, query parameter, or authorization header. Bypass does not invalidate the cached copy. A request for content that is absent from cache after a purge results in `MISS`, not `BYPASS`. |
> | `$upstream_response_length` | `10485760` | Response length from an origin in bytes. In case of multiple responses, commas and colons are used |
> | `$upstream_addr` | `192.0.2.1:80` | IP address and port of the upstream server |
> | `$gcdn_api_client_id` (internal system variable) | `123` | Account ID in the Gcore system |
> | `$gcdn_api_resource_id` (internal system variable) | `01` | CDN resource ID in the Gcore system |
> | `$uid_got` (internal system variable) | `-` | Cookie name and received user ID |
> | `$uid_set` (internal system variable) | `-` | Cookie name and provided user ID |
> | `$geoip_country_code` | `KZ` | Client ISO 3166-1 alpha-2 country code |
> | `$geoip_city` | `-` | Client city code |
> | `$shield_type` (internal system variable) | `shield_no` | Indicates whether [Origin Shielding](/cdn/cdn-resource-options/general/enable-and-configure-origin-shielding) is enabled: `shield_old` – enabled `shield_no` – disabled |
> | `$server_addr` (internal system variable) | `198.51.100.1` | IP address of an Anycast zone or CDN server |
> | `$server_port` (internal system variable) | `80` | Requested port |
> | `$upstream_status` | `206` | Origin response code |
> | `$upstream_connect_time` | `0.000` | Number of seconds (accurate to milliseconds) it took to access an origin server |
> | `$upstream_header_time` | `0.200` | Number of seconds (accurate to milliseconds) it took to receive a response header from an origin server |
> | `$shard_addr` (internal system variable) | `198.51.100.2` | IP address of a CDN server that was first to accept a request if the Cache Sharding feature is enabled |
> | `$geoip2_data_asnumber` | `asnumber` | Number of an autonomous system that sent a request |
> | `$connection` (internal system variable) | `2897494295` | Connection serial number |
> | `$connection_requests` (internal system variable) | `1` | Current number of requests made through a connection |
> | `$http_traceparent` | `00-d5fe1dc9035165ce36952daf29686b6c-14330be33197dd1a-01` | Incoming client `traceparent` request header. If the client omits the header, the field contains `-`; if the value is invalid, the field preserves that value. The CDN-generated replacement is not stored here. See [Traceparent header for troubleshooting](/cdn/troubleshooting/traceparent-header-for-troubleshooting). |
> | `$http_x_forwarded_proto` | `-` | Initial protocol of an incoming request (HTTP or HTTPS) |
> | `$gcdn_internal_status_code` (internal system variable) | `-` | Internal CDN status code. `-` means no internal code was recorded. Other values identify the rule or feature that generated the response. See [internal status codes](#internal-status-codes). |
> | `$ssl_cipher` (internal system variable) | `ECDHE-RSA-AES256-GCM-SHA384` | Cipher name used for an established SSL connection |
> | `$ssl_session_id` (internal system variable) | `28a4184139cb43cdc79006cf2d1a4ac93bdc****` | Session ID of an established SSL connection |
> | `$ssl_session_reused` (internal system variable) | `r` | Shows whether a session was reused (`r`) or not (`.`) |
> | `$sent_http_content_type` | `application/json` | Value of the `Content-Type` HTTP header, indicating the MIME type of a transmitted file |
> | `$tcpinfo_rtt` | `21` | Average time (latency) it takes to transfer a packet to/from a server. The unit of time is microseconds. |
> | `$server_country_code` | `PL` | Server ISO 3166-1 alpha-2 country code |
> | `$gcdn_tcpinfo_snd_cwnd` | `45` | Size of the TCP congestion window: the maximum number of TCP segments the connection can send before an acknowledgment is required |
> | `$gcdn_tcpinfo_total_retrans` | `10` | Total number of retransmitted packets over the life of the connection |
> | `$gcdn_rule_id` | `100700` | Initial rule ID (beta). Possible values are `-` or `100700` |
> | `$request_completion` | `OK` | Whether the CDN finished writing the response body. `OK` means the request completed. An empty value means it did not. Use with `$sent_http_content_length` and `$body_bytes_sent` to detect truncated `200` or `206` responses. |
> | `$sent_http_content_length` | `53214` | `Content-Length` sent to the client: the number of body bytes the CDN declared. Compare with `$body_bytes_sent` to measure truncation. `0` means the header was absent (chunked transfer, no `Content-Length`, or a row from before the field was added), not that zero bytes were sent. |
> | `$gcdn_finalize_reason` | `ok` | Why the request stopped producing a body. `ok` means the body finished. Empty or `unset` means no reason was recorded. Other values identify the cause: client (`client_closed`, `client_timeout`, `client_error`); upstream (`upstream_connect_failed`, `upstream_connect_timeout`, `upstream_headers_null`, `upstream_header_timeout`, `upstream_body_null`, `upstream_body_partial`, `upstream_body_timeout`, `upstream_ssl_error`, `upstream_error`, `upstream_send_timeout`); this proxy (`internal_error`, `shutdown`); or slice (`slice_bad_status`, `slice_missing`, `slice_etag`, `slice_range`). |

### Internal status codes

`$gcdn_internal_status_code` identifies the CDN rule or feature that generated the response.

| Reason | HTTP code | Internal code | Comment |
| :- | :- | :- | :- |
| Country ACL | 403 | 1001 | |
| Referer ACL | 403 | 1002 | |
| IP ACL | 403 | 1003 | |
| User-Agent ACL | 403 | 1004 | |
| Secure Token | 403 | 1005 | Returned when Secure Token validation fails and `$secure_link` is empty |
| Secure Token | 410 | 1006 | Returned when the link lifetime has expired and `$secure_link` is `"0"` |
| WAF | 403 | 1007 | Request blocked by WAF |
| Bot challenge / Testcookie | 307 | 1008 | Redirection sent by bot challenge (testcookie) |
| Blocklist | 403 | 1009 | Request blocked by bot protection blocklist |
| HTTP method | 405 | 1200 | Returned when `AllowedHttpMethods` is enabled and the request method is not in the allowed list |
| Streaming disabled (`.ts` or `.m3u8` files) | 402 | 1201 | Streaming feature disabled for the client |
| LE validation /.well-known/acme-challenge/ | 404 | 1202 | Returned when `http_user_agent` is not `cert-manager-challenges`, `acme.zerossl.com`, `Cpanel-HTTP-Client`, `Buypass validation client`, `Google-Certificates-Bridge`, `vercel-fetch`, `win-acme`, `Typhoeus`, or `Go-http-client` |
| Shield requestor header | 403 | 1203 | Unauthorized request to the shield |
| Force Return | ANY | 1204 | Response generated by the Force Return status code option |

When a Secure Token link passes the authenticity check, `$secure_link` is set to the link extracted from the request URI. That successful validation does not produce a `403` or `410`.

## S3 export

**Amazon S3**

**1. Open Raw Logs**

In the [Gcore Customer Portal](https://portal.gcore.com), navigate to **CDN** > **Logs** > **Raw Logs**.

**2. Enable the export**

Turn on **Receive Raw Logs**. Keep **Do not send empty logs** selected to skip empty files, or clear it to receive empty files.

**3. Include Origin Shielding logs**

If Origin Shielding is enabled, select **Add logs from Origin Shielding** to include edge and pre-cache logs.

**4. Select Amazon S3**

Under **Receive logs using**, select **S3**. Under **Choose S3 storage**, select **Amazon**.

![Amazon S3 export form in Raw Logs settings](https://mintcdn.com/gcore/lClv291c2f00rcjk/images/docs/cdn/logs/raw-logs-export-cdn-resource-logs-to-your-storage/amazon-s3-form.png?fit=max\&auto=format\&n=lClv291c2f00rcjk\&q=85\&s=16b01b2db138bb4f98d7d6455bbff69a)

**5. Enter credentials**

Enter the AWS access key ID and AWS secret access key. See the [Amazon access key guide](https://docs.aws.amazon.com/powershell/latest/userguide/pstools-appendix-sign-up.html).

**6. Set region and buckets**

Optionally choose an AWS region. Then choose one bucket for all CDN resources or a separate bucket per resource, and enter an existing bucket name. Add a folder name to write files inside that bucket.

**7. Save the configuration**

Click **Save changes**.

**Other S3-compatible storage**

**1. Open Raw Logs**

In the [Gcore Customer Portal](https://portal.gcore.com), navigate to **CDN** > **Logs** > **Raw Logs**.

**2. Enable the export**

Turn on **Receive Raw Logs**. Keep **Do not send empty logs** selected to skip empty files, or clear it to receive empty files.

**3. Include Origin Shielding logs**

If Origin Shielding is enabled, select **Add logs from Origin Shielding** to include edge and pre-cache logs.

**4. Select other S3 storage**

Under **Receive logs using**, select **S3**. Under **Choose S3 storage**, select **Other**.

![Other S3 export form in Raw Logs settings](https://mintcdn.com/gcore/lClv291c2f00rcjk/images/docs/cdn/logs/raw-logs-export-cdn-resource-logs-to-your-storage/other-s3-form.png?fit=max\&auto=format\&n=lClv291c2f00rcjk\&q=85\&s=f5718b2c8fe11223a70bf235946da161)

**5. Enter the hostname**

Enter the storage hostname. For Gcore Object Storage, copy the hostname from **Details**.

**6. Enter credentials**

Enter the access key ID and secret access key. For Gcore Object Storage, create the key under [access keys](/storage/manage-object-storage/manage-s3-access-keys) and copy the secret when the key is created.

**7. Enter the bucket hostname**

Enter the bucket hostname in `{bucket_name}.{hostname}` format, using the hostname from the hostname step. For Gcore Object Storage the value looks like `examplename.s-ed1.cloud.gcore.lu`.

**8. Enter the region**

Optionally enter the region (location ID). For Gcore Object Storage this field is required. The location ID is the hostname prefix before the first dot. See [storage Details](/storage/request-content-directly-from-the-storage).

**9. Set buckets**

Choose one bucket for all CDN resources or a separate bucket per resource, and enter an existing bucket name. Add a folder name to write files inside that bucket.

**10. Save the configuration**

Click **Save changes**.

## FTP and SFTP export

**SFTP**

**1. Open Raw Logs**

In the [Gcore Customer Portal](https://portal.gcore.com), navigate to **CDN** > **Logs** > **Raw Logs**.

**2. Enable the export**

Turn on **Receive Raw Logs**. Keep **Do not send empty logs** selected to skip empty files, or clear it to receive empty files.

**3. Include Origin Shielding logs**

If Origin Shielding is enabled, select **Add logs from Origin Shielding** to include edge and pre-cache logs.

**4. Select SFTP**

Under **Receive logs using**, select **SFTP**.

![SFTP export form in Raw Logs settings](https://mintcdn.com/gcore/lClv291c2f00rcjk/images/docs/cdn/logs/raw-logs-export-cdn-resource-logs-to-your-storage/sftp-form.png?fit=max\&auto=format\&n=lClv291c2f00rcjk\&q=85\&s=9ec5403546b11d8070c37ce76f933c8b)

**5. Enter the hostname and port**

Enter `hostname:port`. For Gcore SFTP Storage, copy the hostname from [Details](/storage/request-content-directly-from-the-storage) and append port `2200`, for example `ams.origin.gcdn.co:2200`. For another provider, use that host and its SFTP port.

> [!WARNING]
>
> Omitting the port returns a wrong connection settings error.

**6. Enter credentials**

Enter the **Login** and **Password**. For Gcore SFTP Storage, the login is **Storage/User name** in [Details](/storage/create-an-s3-or-sftp-storage#sftp).

**7. Enter the destination folder**

Optionally enter a **Prepend folder**. Then enter the **Folder**. For Gcore SFTP Storage, use the root folder at the end of the **Upload path** in Details. The home folder is receive-only and cannot be written to directly. The only writable default folder is **htdocs**. For another provider, enter the folder that should receive files, or leave the field empty if the account starts in that folder.

**8. Organize folders**

Choose one folder for all CDN resources or a separate folder per resource. A missing folder name sends files to the root folder.

**9. Save the configuration**

Click **Save changes**.

**FTP**

> [!WARNING]
>
> FTP sends credentials and log data without transport encryption. Use SFTP when logs contain IP addresses, cookies, or request URLs.

**1. Open Raw Logs**

In the [Gcore Customer Portal](https://portal.gcore.com), navigate to **CDN** > **Logs** > **Raw Logs**.

**2. Enable the export**

Turn on **Receive Raw Logs**. Keep **Do not send empty logs** selected to skip empty files, or clear it to receive empty files.

**3. Include Origin Shielding logs**

If Origin Shielding is enabled, select **Add logs from Origin Shielding** to include edge and pre-cache logs.

**4. Select FTP**

Under **Receive logs using**, select **FTP**.

![FTP export form in Raw Logs settings](https://mintcdn.com/gcore/lClv291c2f00rcjk/images/docs/cdn/logs/raw-logs-export-cdn-resource-logs-to-your-storage/ftp-form.png?fit=max\&auto=format\&n=lClv291c2f00rcjk\&q=85\&s=c920d455eb270de465e0336f950414bf)

**5. Enter the hostname and port**

Enter `hostname:port`. Use the FTP host and port from the storage provider, typically port `21`, for example `ftp.example.com:21`.

> [!WARNING]
>
> Omitting the port returns a wrong connection settings error.

**6. Enter credentials**

Enter the **Login** and **Password**.

**7. Enter the destination folder**

Optionally enter a **Prepend folder**. Then enter the **Folder**, or leave it empty if the FTP account starts in that folder.

**8. Organize folders**

Choose one folder for all CDN resources or a separate folder per resource. A missing folder name sends files to the root folder.

**9. Save the configuration**

Click **Save changes**.

## Raw Logs deactivation

**1. Turn off the export**

In the [Gcore Customer Portal](https://portal.gcore.com), navigate to **CDN** > **Logs** > **Raw Logs** and turn off **Receive Raw Logs**.

**2. Save the change**

Click **Save changes**. Delivery stops only after this save. The stored S3, FTP, or SFTP settings remain and apply again when **Receive Raw Logs** is turned on and saved.

![Raw Logs page with Receive Raw Logs turned off](https://mintcdn.com/gcore/lClv291c2f00rcjk/images/docs/cdn/logs/raw-logs-export-cdn-resource-logs-to-your-storage/raw-logs-page-off.png?fit=max\&auto=format\&n=lClv291c2f00rcjk\&q=85\&s=873da67cb012ef61f39427c1494bf0cc)

**Discard changes** restores the last saved configuration. If **Receive Raw Logs** was on when the page was last saved, **Discard changes** turns the toggle back on and does not deactivate the export.