aboutsummaryrefslogtreecommitdiff
path: root/crates/tor-dirserver/src
Commit message (Collapse)AuthorAgeFilesLines
...
| * database: Introduce DocumentIdClara Engler2026-01-195-136/+210
| | | | | | | | | | | | This commit introduces a new type in the database module named `DocumentId` for abstracting the underlying content-addressable hash algorithm we are using.
* | tor-dirserver: Make downloader statelessClara Engler2026-01-202-53/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit makes the downloader found in `mirror::operation::download` stateless by removing the `preferred_authority` field and changing all methods from `&mut self` to `&self`. The preferred authority is now accepted as a parameter to `DownloadManager::download` which also returns a tuple now with the actually used authority alongside the response, putting the management of this to the responsibility of the caller. Meanwhile, it also renames the structure from `ConsensusBoundDownloader` to `DownloadManager` because it no longer keeps a state that invalidates after a consensus "ends". The purpose of this is to simplify overall state in order to make the implementation of a finite-state-machine for the dirmirror operation more easy (and deterministic).
* | Fix flaky `request_fail_ultimately` testClara Engler2026-01-191-1/+10
|/ | | | | | | | | | | | | This commit fixes the flaky `request_fail_ultimately` in tor-dirserver, which is flaky due to the operating system's handling of TCP RSTs, which are generally detected stochastically and sometimes are not received even after the entire response has been parsed. This leads to tor-dirclient either returning a connection reset or a truncated header error, depending on whether it successfully or unsuccessfully reads zero bytes from the server. Fixes #2318
* tor-dirserver: Small pre-merge fixesClara Engler2026-01-152-3/+2
|
* tor-dirserver: Add TODO wrt hex::encode_upperClara Engler2026-01-151-0/+2
| | | | | Adding a comment suggesting to implement this as part of abstracting `Sha256` behind a more generic identifier.
* tor-dirserver: Fix clippy warningClara Engler2026-01-151-1/+1
|
* tor-dirserver: Use `INSERT OR REPLACE`Clara Engler2026-01-151-2/+43
| | | | | | | | | | | This commit rpelaces the use of `INSERT` with `INSERT OR REPLACE` for handling insertion conflicts in `store_insert`. Because everything is content-addressed anyways, it does not matter to do this, because if we get a conflict on a SHA256, it means the data is equal.[1] [1]: Excluding SHA-2 collisions which are impractical as of 2026.
* tor-dirserver: Change store_insert logicClara Engler2026-01-152-49/+53
| | | | | | This commit modifies the database::store_insert logic to accept an iterator of the encodings to store the document in, instead of encoding it with all encodings we support.
* tor-dirserver: Remove algorithm CHECK constraintClara Engler2026-01-151-2/+1
|
* tor-dirserver: Add TODO for hash agnostic type alias nameClara Engler2026-01-151-0/+1
|
* tor-dirmirror: Handle unparsable certificatesClara Engler2026-01-151-8/+69
| | | | Instead of failing, add them to missing and query them again.
* tor-dirserver: Improve note on query performanceClara Engler2026-01-151-6/+8
|
* tor-dirserver: Fix formulationClara Engler2026-01-151-1/+1
|
* tor-dirserver: Implement authcert retrieval logicClara Engler2026-01-152-3/+279
| | | | | | | | | | | This commit implements the logic necessary to retrieve the (missing) directory authority certificates from an upstream directory authority. In order to do so, this commit implements three new functions: 1. `download_authority_certificates()` 2. `parse_authority_certificates()` 3. `verify_authority_certificates()` 4. `insert_authority_certificates()`
* tor-dirserver: Ensure algorithm with CHECKClara Engler2026-01-151-2/+3
| | | | | | This commit adds a CHECK constraint to the compressed_document table in order to ensure that `algorithm` may only take up a limited set of values.
* tor-dirserver: Move ContentEncoding to databaseClara Engler2026-01-152-20/+18
| | | | | | This commit moves the ContentEncoding enum from the http module to the database module, primarily because the content encoding is more of a matter to the database, as this is where the data actually resides.
* tor-dirserver: Support for inserting into storeClara Engler2026-01-153-5/+222
| | | | | | | | This commit implements support for adding arbitrary documents into the store table. It is non-trivial because the relevant data has to be compressed into various formats, so it can be retrieved without delays.
* tor-dirserver: Test authcert queryingClara Engler2026-01-151-0/+92
|
* tor-dirserver: Operation test refactoringClara Engler2026-01-151-9/+7
|
* tor-dirserver: Add get_recent_authority_certificatesClara Engler2026-01-151-2/+110
| | | | | | | This commit adds the `get_recent_authority_certificates()` function to the operation of a directory mirror, which is responsible for looking up the certificates in the database, returning the parsed found ones as well as the missing ones.
* tor-dirserver: `From<Timestamp> for SystemTime`Clara Engler2026-01-151-0/+6
| | | | | | The reverse direction already exists and it will be required when we act with functions outside the crate that only accept a `SystemTime` in situations where we just have a `Timestamp`.
* tor-dirserver: Add `dir_key_published` to schemaClara Engler2026-01-151-1/+4
|
* tor-dirserver: Speedup tests using tokio's test-utilClara Engler2026-01-141-8/+6
| | | | | | | | | | | | | | | | | | | | | This commit speeds up the tests in `tor-dirserver` by using functionality from Tokio's `test-util`. Most notably, it runs the time intensive test in paused mode, which means that the clock gets advanced either explicitly or implicitly using auto-advance in case the runtime has nothing to do. In our case, the last one, auto-advancing, is used, leading to our sleep calls returning immediately. This is good enough for testing in this module. It is not the responsibility of tor-dirserver to ensure whether `RetryDelay` returns proper values, as this is the responsibility of `tor-basic-utils`. Still, it is a bit unfortunate that Tokio offers no way to manually disbale the auto-advancing. In the future, it might be useful to migrate more parts of this to crate to `tor-rtcompat`, but for now, this change is a good enough performance fix.
* tor-dirserver: Remove slow ineffective testClara Engler2026-01-141-27/+0
| | | | | | | | | | This commit removes `mirror::download::test::request_fail_timeout` which takes more than five seconds (due to a timeout) and is generally ineffective in what it does, because testing whether a task is still sleeping after a certain time is not super trivial with the tools Tokio offers. All this test offers is testing that we enter a timeout at all.
* Merge branch 'flaky-busy-timeout' into 'main'Ian Jackson2026-01-141-38/+58
|\ | | | | | | | | | | | | tor-dirserver: Eliminate use of sleep in tests Closes #2308 See merge request tpo/core/arti!3582
| * tor-dirserver: Eliminate use of sleep in testsClara Engler2026-01-131-38/+58
| | | | | | | | | | | | | | | | | | | | | | | | This commit eliminates the use of sleep in database tests because those can be flaky, especially when the CI scheduler is overloaded, leading to potential timeout invariants not holding true anymore. We now fix this by using synchronization primitives from Rust in order for threads to communicate. Documentation has been added explaining how these tests work in greater detail. Fixes #2308
* | tor-dirserver: Fix code broken by rusqlite upgradeClara Engler2026-01-141-2/+10
|/ | | | | | | SQLite does not support u64 and the new version of SQLite makes a certain piece of tor-dirserver no longer compile due to the lack of `ToSql` for `u64`. This commit fixes it by converting these types from `u64` to `i64` saturatingly.
* fix(download): Use push_timed with mockable time sourcesNihal2025-12-171-2/+2
|
* tor-dirserver: Document nested round-robinsClara Engler2025-12-101-2/+15
|
* tor-dirserver: Improve download_single's docClara Engler2025-12-101-5/+2
|
* tor-dirserver: Rename DownloadManager to ConsensusBoundDownloaderClara Engler2025-12-101-13/+13
|
* tor-dirclient: Remove Downloadable for goodClara Engler2025-12-101-4/+4
|
* tor-dirserver: Add doc for TcpConnect errorClara Engler2025-12-101-0/+4
|
* tor-dirserver: Documentation improvementsClara Engler2025-12-101-3/+5
|
* tor-dirserver: Replace IO with TcpConnect errorClara Engler2025-12-102-4/+18
| | | | | | | | This commit replaces `AuthorityCommunicationError::IO` with `AuthorityCommunicationError::TcpConnect` while removing the respective `From<std::io::Error>` implementation, mostly because an IO error is way too generic and in our case, it would only occurr with a failed TCP connection.
* tor-dirserver: Fix wrong warning sectionClara Engler2025-12-101-1/+1
|
* tor-dirserver: Implement DownloadManagerClara Engler2025-12-103-0/+397
| | | | | | | | | This commit implements the DownloadManager in its own module, an object responsible for downloading network documents from an upstream directory authority. Further motivation for this can be found in the rustdoc comment of the respectively introduced type.
* tor-dirserver: Improve database::TimestampClara Engler2025-11-272-64/+99
| | | | | | | | | | This commit improves upon the work made in arti!3460, by establishing the `database::Timestamp` type as a general purpose type for the handling of timestamp throughout the code base. It also replaces all previous occurrences of `SystemTime` within `dirmirror::operation`, which leads to the reduction of code complexity in one particular function, namely `calculate_sync_timeout`.
* tor-dirserver: Fix clippy warning for timeClara Engler2025-11-261-0/+4
|
* tor-dirserver: Use `saturating-time`Clara Engler2025-11-262-243/+90
| | | | | | | This commit replaces the custom implementation of `SaturatingSystemTime` with the `saturating-time` crate, additionally adding a new type to the `database` module called `Timestamp`, in order to have a convenient wrapper around `ToSql` and `FromSql`.
* Fix name of clippy lint to unchecked_time_subtraction (2)Ian Jackson2025-11-065-5/+5
| | | | Run maint/add_warning
* Merge branch 'dirserver-http-db' into 'main'Ian Jackson2025-11-052-120/+114
|\ | | | | | | | | tor-dirserver: Apply database changes to http See merge request tpo/core/arti!3426
| * tor-dirserver: clippy fixesClara Engler2025-11-032-4/+4
| |
| * tor-dirserver: Apply database changes to httpClara Engler2025-11-032-120/+114
| | | | | | | | | | | | | | | | | | | | This commit applies the recent changes to the database module into the http and http::cache module. Notably, this includes the use of the database::sql! macro as well as the use of database::read_tx and database::rw_tx for creation transaction and saving, as well as retrying, although the latter one is subject to SQLite's own busy handler
* | tor-dirserver: Refactor cons retrieval with JOINClara Engler2025-11-041-19/+16
|/ | | | | | | | | | | | This commit refactors `get_recent_consensus()` in `tor-dirserver` to use a single SQL statement with the power of `INNER JOIN` instead of two SQL statements querying the store after retrieving the SHA256 hash. The primary motivation for this is to have cleaner code. A single SQL statement is often-times easier to comprehend, as well more performant in execution, although that can only be proven with a profiler and is not of much concern, given that this function is only executed roughly every 60-90min.
* tor-dirserver: Add SaturatingSystemTimeClara Engler2025-10-302-156/+288
| | | | | | This commit changes various things in the dirmirror operation, namely the use of a SystemTime wrapper type that provides saturation on both ends.
* tor-dirserver: Ensure positive timestampsClara Engler2025-10-301-1/+5
| | | | | This commit adds `CHECK` constraints to the database schema in order to ensure all timestamps are positive.
* tor-dirserver: Small database::open() improvementsClara Engler2025-10-301-115/+58
|
* tor-dirserver: Make database full syncClara Engler2025-10-305-289/+229
| | | | | | | | | | This commit makes the entire database operations synchronous, thereby replacing deadpool with r2d2. The full motivation is outlined in a rustdoc comment at the top of the `database` module, but it can be summarized to the fact that SQLite is by design inherently synchronous due to directly interfacing with the file system.
* tor-dirserver: Outline the dirmirror operationClara Engler2025-10-302-147/+260
| | | | | | | | | | This commit creates a function named `mirror::operation::serve`, that will serve as the primary entry point for the dirmirror operation. While the function itself already contains some logic, such as querying the timeout to wait for the next consensus, it is largely a rustdoc design document that adds lots of comment to the `mirror::operation` module.
generated by cgit v1.2.3 (git 2.46.0) at 2026-08-27 16:47:18 +0000