Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ This is a great place to meet other contributors and get guidance on where to co
However, all technical designs should also be recorded and formalized in GitHub issues, so that they are accessible to everyone.
In Slack, find us in the `#arrow-rust` channel and feel free to ask for an invite via Discord, GitHub issues, or other means.

There is more information in the [contributing] guide.
There is more information in the [contributing] guide and the [security] policy.

## Repository Structure

Expand Down Expand Up @@ -186,3 +186,4 @@ You can find more details about each crate in their respective READMEs.
[issues]: https://github.com/apache/arrow-rs/issues
[pull requests]: https://github.com/apache/arrow-rs/pulls
[discussions]: https://github.com/apache/arrow-rs/discussions
[security]: SECURITY.md
49 changes: 49 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
<!---
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Security Policy

This document outlines the security model for the Rust implementation of Apache Arrow (`arrow-rs`) and how to report vulnerabilities.

## Security Model

The `arrow-rs` project follows the [Apache Arrow Security Model]. Key aspects include:
- Reading data from untrusted sources (e.g., over a network or from a file) requires explicit validation.
- Failure to validate untrusted data before use may lead to security issues. This implementation provides APIs to validate Arrow data. For example, [`ArrayData::validate_full`] can be used to ensure that data conforms to the Arrow specification.

## Rust Safety and Undefined Behavior
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should define what exploitable means, as it is so important to this section.

Perhaps inline whatever gets merged from https://github.com/apache/arrow/pull/49761/changes or at least link to it


We strive to uphold the [Rust Soundness Pledge].

- **Undefined Behavior (UB) is a bug:** Any instance of UB is a bug we are committed to fixing.
- **UB as a Security Issue:** Any **exploitable** UB triggered via safe APIs is a security issue. Other UB instances are bugs, and we welcome help fixing them.

## Reporting a Vulnerability

**Do not file a public issue.** Follow the [ASF security reporting process] by emailing [security@apache.org](mailto:security@apache.org).

Include in your report:
- A clear description and minimal reproducer.
- Affected crates and versions.
- Potential impact.

[Apache Arrow Security Model]: https://arrow.apache.org/docs/dev/format/Security.html
[`ArrayData::validate_full`]: https://docs.rs/arrow/latest/arrow/array/struct.ArrayData.html#method.validate_full
[Rust Soundness Pledge]: https://raphlinus.github.io/rust/2020/01/18/soundness-pledge.html
[ASF security reporting process]: https://www.apache.org/security/#reporting-a-vulnerability
6 changes: 4 additions & 2 deletions arrow-avro/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,9 +212,11 @@ async fn main() -> anyhow::Result<()> {
* **Confluent Schema Registry wire format**: 1‑byte magic `0x00` + 4‑byte BE schema ID + Avro body; supports decode + encode helpers.
* **Avro Single‑Object Encoding (SOE)**: 2‑byte magic `0xC3 0x01` + 8‑byte LE CRC‑64‑AVRO fingerprint + Avro body; supports decode + encode helpers.

---
## Security

See the [Security Policy] for information on the security model and how to report vulnerabilities.

## Examples
[Security Policy]: https://github.com/apache/arrow-rs/blob/main/SECURITY.md

* Read/write OCF in memory and from files (see crate docs “OCF round‑trip”).
* Confluent wire‑format and SOE quickstarts are provided as runnable snippets in docs.
Expand Down
33 changes: 33 additions & 0 deletions arrow-csv/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
<!---
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# `arrow-csv`

Support for reading/writing CSV files to/from [Apache Arrow].

See the [main repository README] and the [API documentation] for more details.

## Security

See the [Security Policy] for information on the security model and how to report vulnerabilities.

[Apache Arrow]: https://arrow.apache.org/
[main repository README]: https://github.com/apache/arrow-rs
[API documentation]: https://docs.rs/arrow-csv/latest
[Security Policy]: https://github.com/apache/arrow-rs/blob/main/SECURITY.md
6 changes: 5 additions & 1 deletion arrow-flight/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,4 +81,8 @@ $ flight_sql_client --host example.com statement-query "SELECT 1;"
+----------+
```

[apache arrow flightsql]: https://arrow.apache.org/docs/format/FlightSql.html
## Security

See the [Security Policy] for information on the security model and how to report vulnerabilities.

[security policy]: https://github.com/apache/arrow-rs/blob/main/SECURITY.md
34 changes: 34 additions & 0 deletions arrow-ipc/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
<!---
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# `arrow-ipc`

Support for reading/writing files and streams of the [Arrow IPC Format] to/from [Apache Arrow].

See the [main repository README] and the [API documentation] for more details.

## Security

See the [Security Policy] for information on the security model and how to report vulnerabilities.

[Apache Arrow]: https://arrow.apache.org/
[Arrow IPC Format]: https://arrow.apache.org/docs/format/Columnar.html#format-ipc
[main repository README]: https://github.com/apache/arrow-rs
[API documentation]: https://docs.rs/arrow-ipc/latest
[Security Policy]: https://github.com/apache/arrow-rs/blob/main/SECURITY.md
33 changes: 33 additions & 0 deletions arrow-json/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
<!---
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# `arrow-json`

Support for reading and writing JSON to/from [Apache Arrow].

See the [main repository README] and the [API documentation] for more details.

## Security

See the [Security Policy] for information on the security model and how to report vulnerabilities.

[Apache Arrow]: https://arrow.apache.org/
[main repository README]: https://github.com/apache/arrow-rs
[API documentation]: https://docs.rs/arrow-json/latest
[Security Policy]: https://github.com/apache/arrow-rs/blob/main/SECURITY.md
33 changes: 11 additions & 22 deletions arrow/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,32 +76,21 @@ The `arrow` crate provides the following features which may be enabled in your `

The [Apache Arrow Status](https://arrow.apache.org/docs/status.html) page lists which features of Arrow this crate supports.

## Safety

Arrow seeks to uphold the Rust Soundness Pledge as articulated eloquently [here](https://raphlinus.github.io/rust/2020/01/18/soundness-pledge.html). Specifically:
## Safety and Security

> The intent of this crate is to be free of soundness bugs. The developers will do their best to avoid them, and welcome help in analyzing and fixing them
`arrow-rs` follows the [Apache Arrow Security Model]. Any **exploitable** instance of undefined behavior (UB) triggered via safe APIs is a security issue. See our [Security Policy] for reporting.

Where soundness in turn is defined as:
We uphold the [Rust Soundness Pledge], aiming to be free of UB from safe APIs. While `unsafe` is used for performance or FFI, we mitigate risk through:
- Strongly-typed `Array` and `ArrayBuilder` APIs.
- Extensive `ArrayData` validation for untrusted sources.
- [MIRI] verification in CI.
- A `force_validate` feature for extra checks.

> Code is unable to trigger undefined behavior using safe APIs

One way to ensure this would be to not use `unsafe`, however, as described in the opening chapter of the [Rustonomicon](https://doc.rust-lang.org/nomicon/meet-safe-and-unsafe.html) this is not a requirement, and flexibility in this regard is one of Rust's great strengths.

In particular there are a number of scenarios where `unsafe` is largely unavoidable:

- Invariants that cannot be statically verified by the compiler and unlock non-trivial performance wins, e.g. values in a StringArray are UTF-8, [TrustedLen](https://doc.rust-lang.org/std/iter/trait.TrustedLen.html) iterators, etc...
- FFI

Additionally, this crate exposes a number of `unsafe` APIs, allowing downstream crates to explicitly opt-out of potentially expensive invariant checking where appropriate.

We have a number of strategies to help reduce this risk:

- Provide strongly-typed `Array` and `ArrayBuilder` APIs to safely and efficiently interact with arrays
- Extensive validation logic to safely construct `ArrayData` from untrusted sources
- All commits are verified using [MIRI](https://github.com/rust-lang/miri) to detect undefined behaviour
- Use a `force_validate` feature that enables additional validation checks for use in test/debug builds
- There is ongoing work to reduce and better document the use of unsafe, and we welcome contributions in this space
[Rust Soundness Pledge]: https://raphlinus.github.io/rust/2020/01/18/soundness-pledge.html
[MIRI]: https://github.com/rust-lang/miri
[Apache Arrow Security Model]: https://arrow.apache.org/docs/dev/format/Security.html
[Security Policy]: https://github.com/apache/arrow-rs/blob/main/SECURITY.md

## Building for WASM

Expand Down
22 changes: 16 additions & 6 deletions arrow/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -335,14 +335,24 @@
//! * [`parquet`](https://docs.rs/parquet) - support for [Apache Parquet]
//! * [`arrow-avro`](https://docs.rs/arrow-avro) - support for [Apache Avro]
//!
//! # Safety and Security
//! # Security
//!
//! Like many crates, this crate makes use of unsafe where prudent. However, it endeavours to be
//! sound. Specifically, **it should not be possible to trigger undefined behaviour using safe APIs.**
//! This project follows the [Apache Arrow Security Model]. Any exploitable
//! instance of undefined behavior using `safe` APIs and having a clear explanation
//! or reproducer is considered a security issue.
//!
//! If you think you have found an instance where this is possible, please file
//! a ticket in our [issue tracker] and it will be triaged and fixed. For more information on
//! arrow's use of unsafe, see [here](https://github.com/apache/arrow-rs/tree/main/arrow#safety).
//! If you think you have found a security vulnerability or a soundness bug,
//! please follow the instructions in our [security policy] for reporting.
//!
//! # Safety
//!
//! Like many crates, this crate makes use of `unsafe` where prudent. However, it endeavors to be
//! sound. Specifically, **it should not be possible to trigger undefined behavior using safe APIs.**
//!
//! For more information on the use of unsafe, see [here](https://github.com/apache/arrow-rs/tree/main/arrow#safety-and-security).
//!
//! [Apache Arrow Security Model]: https://arrow.apache.org/docs/dev/format/Security.html
//! [security policy]: https://github.com/apache/arrow-rs/blob/main/SECURITY.md
//!
//! # Higher-level Processing
//!
Expand Down
6 changes: 6 additions & 0 deletions parquet/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,12 @@ information on the status of this implementation.
[implementation status page]: https://parquet.apache.org/docs/file-format/implementationstatus/
[apache parquet]: https://parquet.apache.org/

## Security

See the [Security Policy] for information on the security model and how to report vulnerabilities.

[security policy]: https://github.com/apache/arrow-rs/blob/main/SECURITY.md

## License

Licensed under the Apache License, Version 2.0: <http://www.apache.org/licenses/LICENSE-2.0>.
6 changes: 6 additions & 0 deletions parquet_derive/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,12 @@ To compile and test doctests, run `cargo test --doc -- --show-output`
To build documentation, run `cargo doc --no-deps`.
To compile and view in the browser, run `cargo doc --no-deps --open`.

## Security

See the [Security Policy] for information on the security model and how to report vulnerabilities.

[Security Policy]: https://github.com/apache/arrow-rs/blob/main/SECURITY.md

## License

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0.
Loading