summaryrefslogtreecommitdiff
path: root/crates/tor-dirserver/src/database.rs
Commit message (Collapse)AuthorAgeFilesLines
* Much formatting churn for 2024 editionIan Jackson2026-07-221-35/+42
|
* maint: Run maint/add_warning to deny string slicesClara Engler2026-06-091-0/+1
| | | | | | | | | | | | This commit executes maint/add_warning with the just added change to deny string slices except in tests. I recommend auditing this by checking out the previous commit followed by running the script yourself and then verifying that the diff is identical to this commit. This commit makes cargo clippy fail. We will add exceptions in the next commit.
* tor-dirserver: Fix for missing_extra_infos() for NULLClara Engler2026-04-281-0/+1
| | | | | | | | | | If extra-infos is set to NULL, which might be the case for micro descriptors or even router descriptors because the field is optional there, this SQL query fails because it cannot LEFT JOIN server.extra_unsigned_sha1 when this field is NULL. To fix this, we simply add an additional clause to the WHERE statement that filters such rows out.
* tor-dirserver: Make use of only one descriptor hash in missing selectionClara Engler2026-04-271-0/+2
| | | | | | | | | | | | | | | This commit makes use of the presence of only one descriptor hash inside consensus_router_descriptor_member by adding a respective AND clause to the select statements of missing descriptors to only select the rows with a non-null SHA-1 (or SHA-2 for the sake of microdescs) in the consensus_router_descriptor_member table. This is defensive programming and should not be required in practice because the docid as well as the "XOR" CHECK should already ensure that this value is always non-NULL. However, if it still happens and slips through, the statement would return NULL rows, which is not what we want and was previously prevented by ensuring this field was never NULL in the first place.
* tor-dirserver: Store precisely one descriptor hash in schemaClara Engler2026-04-271-8/+4
| | | | | | | | | | | | | | | | | | This commit modifies the consensus_router_descriptor_member table in the database schema, removing the NOT NULL constraint on unsigned_sha1 and unsigned_sha2 by replacing it with a new CHECK constraint that checks that either one of them is set but not both. The reason for this is as follows: We are going to use this table to compute the queue of missing descriptors, which means that we can only populate this table with the data we know from the consensus. The consensus however tells us only one of those hashes, namely sha1 in the case of a consensus-ns and sha2 in the case of a consensus-md. In other words: This commit can also be seen as an effort to change the design of the operation in such a way that the queue is obtained directly from the database and not computed at the start during state transition.
* dirserver: Skip warnings re SaturatingTime::saturating_duration_sinceNick Mathewson2026-04-211-0/+2
| | | | | Rust 1.95 warns us that this is an experimental API that could be stabilized in the future.
* Fix typosTobias Stoeckmann2026-03-241-8/+8
| | | | Typos found with codespell
* tor-dirserver: Use collect instead of mutable forClara Engler2026-03-091-18/+9
|
* tor-dirserver: Use tor-llcrypto for digestsClara Engler2026-03-091-4/+4
| | | | | | | | This commit replaces the uses of sha1, sha2, and sha3 with their respective pedants from tor-llcrypto for better consistency. Internally, they still use the same logic and underlying crates but let's use this encapsulation nonetheless.
* tor-dirserver: Improve hash wrapper macro docsClara Engler2026-03-091-0/+19
|
* tor-dirserver: Move schema to own fileClara Engler2026-03-091-170/+1
| | | | | | | | This commit moves the database schema into schema_v1.sql and uses include_str to include it. For now, the schema lives in the `src/` directory. If this becomes a problem because we get more schemas and schema upgrades, we can move it into another sub directory, but let's not overengineer the hierarchy there.
* tor-dirserver: Implement `State::AuthCerts`Clara Engler2026-03-091-1/+45
| | | | | | | | | | | | | | This commit implements the logic required for retrieving, validating, and storing authority certificates. The implementation determines the missing certificates by looking at the signatories of the unvalidated consensus and checking them in the db. Afterwards, they will be queried and individually filtered and verified before being inserted into the database. A return of this implementation notably DOES NOT imply all missing certificates have been downloaded. This was chosen for a simplified retry logic.
* tor-dirserver: Fix rustdoc commentsClara Engler2026-03-091-4/+4
|
* tor-dirserver: Query missing descriptor logicClara Engler2026-03-091-1/+394
| | | | | | | | | | | | | | | | | | | This commit implements the logic for querying missing descriptors by adding three new methods to ConsensusMeta: * missing_servers * missing_extras * missing_micros All of them essentially work the same, namely by querying the consensus_router_descriptor_member database table alongside the docid of the current consensus, left joining the respective router_descriptor and returning the digests on all rows where the left join resulteed in a NULL. A small exception are extra-info descriptors. There, we can only return the ones where we have an accompanying server descriptor, hence why we perform an inner join with the server descriptors between the from and the left join.
* tor-dirserver: Rename `md` to `microdesc`Clara Engler2026-03-091-2/+2
| | | | | This is better because it is returned by `ConsensusFlavor::Microdesc::name()`.
* tor-dirserver: Fix extra info schemaClara Engler2026-03-091-7/+10
| | | | | | | | | | | | This commit fixes some parts in the extra-info related part of the database schema. Most notably, it changes the semantic to indicate that the sha1 represents the unsigned part of the extra-info document. Likewise, it also ensures that the reference to an extra-info document must not be null with a plain consensus but null with a microdescriptor consensus.
* tor-dirserver: Store unsigned SHA for descsClara Engler2026-03-091-12/+13
| | | | | | | | | | | This commit modifies the database schema to rename `sha1` and `sha2` to `unsigned_sha1` and `unsigned_sha2`. Network status consensuses refer to server descriptors by their unsigned sha1. Microdescriptor consensuses refer to microdescriptors by their sha2, which is always unsigned, so we adapt that name for consistency.
* tor-dirserver: Remove foreign key from descriptor memberClara Engler2026-03-091-4/+8
| | | | | | | | | | | This commit removes the foreign key constraints from the `consensus_router_descriptor_member` in order to allow the insertion of the router descriptors contained in a consensus before the respective descriptors were fetched. This also allows to directly compute the missing descriptors in SQL, using a left join on this table on `router_descriptor` and obtaining the entries that are NULL.
* tor-dirserver: Store sha1 for extra-infoClara Engler2026-03-091-2/+6
| | | | | | | | | | | This commit modifies the database schema by tracking the sha1 of the extra-info instead of the rowid. This makes generating queues and other things easier. It also removes the foreign key constraint in order to allow for an asynchronous retrieval and storage. Otherwise it would not be possible to store a router descriptor without having obtained the extra info first.
* tor-dirserver: Use getters for meta structsClara Engler2026-03-091-13/+16
| | | | | | This approach is better in order to guarantee that invariants are not violated, such that the expiry timestamp not being earlier than the valid after one for example.
* tor-dirserver: Derive Copy on database meta structsClara Engler2026-03-091-2/+2
|
* tor-dirserver: Implement Sha1Clara Engler2026-03-091-4/+11
| | | | | This commit implements Sha1 for the database in order to use it for the fingerprints in authority key certificates.
* tor-dirserver: Implement Sha3_256Clara Engler2026-03-091-8/+5
| | | | | This commit implements Sha3_256 as a database type and uses it for stroing `ConsensusMeta::unsigned_sha3_256`.
* tor-dirserver: Generic database hash implementationClara Engler2026-03-091-65/+69
| | | | | | | | | | | | | | | This commit moves the `DocumentId` implementation into a macro called `impl_hash_wrapper` that implements a hash type in a database compatible fashion. In our case, we implement this for `sha2::Sha256` and then type alias `DocumentId` to this new hash. Yes, we originally moved away from a type alias here, but I think this is fine because for the foreseeable future, we will continue to use a hash here, just maybe not Sha2, but the flexibility remains. The motivation for this is to support other hash algorithms similarly too.
* tor-dirserver: pub(crate) for ConsensusMeta::lifetimeClara Engler2026-03-091-1/+1
|
* tor-dirserver: Move AuthCert to databaseClara Engler2026-03-091-1/+256
| | | | | | | | | This commit moves get_recent_auth_certs from operation to database by introducing a new struct called `AuthCertMeta` containing the database metadata alongside an accompanying data method returning the raw data. For now, it leaves out the download, verify, and insert logic. We will add that back later once we will need it.
* tor-dirserver: Rename Consensus to ConsensusMetaClara Engler2026-03-091-23/+31
| | | | | | | This commit renames the database `Consensus` to `ConsensusMeta` in order to not collide with the naming from tor-netdoc and to clearly indicate that this data is just metadata about such a document, but not the document itself.
* tor-dirserver: Avoid super in testsClara Engler2026-03-091-54/+44
| | | | | If is annoying to use, especially if we can avoid it by just avoid name conflicts.
* tor-dirserver: Move db constants to the topClara Engler2026-03-091-172/+172
| | | | | | | Right now, it is a bit unfortunate that we have the very long schema string between two code blocks. It is probably better to either have it at the top or the bottom of the file, hence why this commit moves it to the top.
* tor-dirserver: Move sync timeout to databaseClara Engler2026-03-091-0/+36
| | | | | This commit moves calculate_sync_timeout into the Consensus struct in the database module, as it fits better there.
* tor-dirserver: Refactor consensus retrieval logicClara Engler2026-03-091-1/+291
| | | | | | | | This commit moves the get_recent_consensus logic to the database module, which also introduces a struct querying all fields in it. This not only simplifies the return type but also makes working with this type more comfortable to work with.
* tor-dirserver: docid as consistent foreign keyClara Engler2026-03-091-10/+10
| | | | | | This commit replaces occurrences of rowid with docid, particularly in tables for N:M cardinalities. Purpose of this is, to achieve a greater consistency with the overall database schema.
* tor-dirserver: Add comment on compression bugClara Engler2026-01-221-0/+4
| | | | | This commit adds a comment explaining why we treat compression errors as a bug.
* tor-dirserver: Make compression failures a bugClara Engler2026-01-221-1/+2
| | | | | | | | | | | | | | This commit removes `DatabaseError::Compression` because it does not fit in. Right now, this single variant makes the error to be call-site oriented which is not nice for error handling. Instead, this error should indicate that something was truly wrong with the database in itself, such as an invalid schema, a low-level SQLite bug, etc. Instead, we now map a compression error to `DatabaseError::Bug` because there is no good reason on why it should fail, given that we compress memory data to memory data. Probably because it uses the `std::io::Writer` interface which itself demands use of `std::io::Result`.
* tor-dirserver: Remove FromStr for DocumentIdClara Engler2026-01-221-10/+5
| | | | | | | | | | | This commit removes FromStr for DocumentId because it was only used in testing anyways. Instead, it replaces it with a simple From<[u8; 32]> only enabled in test builds, which is sufficient for what we are trying to do. An alternative would be to make the inner field pub, but this seems to aggressive for a testing only thing.
* tor-dirserver: Add `sha2` to `router_descriptor`Clara Engler2026-01-191-0/+2
| | | | | | | | | | This comit adds a `sha2` column to `router_descriptor` alongside a `CHECK` to see whether it equals `docid`. The reason for this is simple: Microdescriptors are the only kind of documents that are queriable with a SHA2 hash. Previously, we would have simply used the `docid` column for this, but in order to abstract it better, a distinct column with this hash is better.
* tor-dirserver: Rename `doc_id` to `docid`Clara Engler2026-01-191-14/+13
| | | | | Because we went with `docid` in the database (due to `rowid`), it is only natural to call the code variables `docid` too.
* tor-dirserver: Rename sha256 to docid in schemaClara Engler2026-01-191-31/+31
| | | | | | This commit renames the sha256 column to docid for the reason that we agreed upon making the schema visible to all modules, so if we were to encapsulate docid properly, this change is only natural.
* tor-dirserver: Rename database meta tableClara Engler2026-01-191-16/+13
| | | | | This commit renames arti_dirmirror_schema_version to arti_dirserver_schema_version.
* database: Introduce DocumentIdClara Engler2026-01-191-23/+89
| | | | | | This commit introduces a new type in the database module named `DocumentId` for abstracting the underlying content-addressable hash algorithm we are using.
* tor-dirserver: Small pre-merge fixesClara Engler2026-01-151-1/+0
|
* tor-dirserver: Add TODO wrt hex::encode_upperClara Engler2026-01-151-0/+2
| | | | | Adding a comment suggesting to implement this as part of abstracting `Sha256` behind a more generic identifier.
* tor-dirserver: Fix clippy warningClara Engler2026-01-151-1/+1
|
* tor-dirserver: Use `INSERT OR REPLACE`Clara Engler2026-01-151-2/+43
| | | | | | | | | | | This commit rpelaces the use of `INSERT` with `INSERT OR REPLACE` for handling insertion conflicts in `store_insert`. Because everything is content-addressed anyways, it does not matter to do this, because if we get a conflict on a SHA256, it means the data is equal.[1] [1]: Excluding SHA-2 collisions which are impractical as of 2026.
* tor-dirserver: Change store_insert logicClara Engler2026-01-151-47/+45
| | | | | | This commit modifies the database::store_insert logic to accept an iterator of the encodings to store the document in, instead of encoding it with all encodings we support.
* tor-dirserver: Remove algorithm CHECK constraintClara Engler2026-01-151-2/+1
|
* tor-dirserver: Add TODO for hash agnostic type alias nameClara Engler2026-01-151-0/+1
|
* tor-dirserver: Ensure algorithm with CHECKClara Engler2026-01-151-2/+3
| | | | | | This commit adds a CHECK constraint to the compressed_document table in order to ensure that `algorithm` may only take up a limited set of values.
* tor-dirserver: Move ContentEncoding to databaseClara Engler2026-01-151-1/+17
| | | | | | This commit moves the ContentEncoding enum from the http module to the database module, primarily because the content encoding is more of a matter to the database, as this is where the data actually resides.
* tor-dirserver: Support for inserting into storeClara Engler2026-01-151-3/+215
| | | | | | | | This commit implements support for adding arbitrary documents into the store table. It is non-trivial because the relevant data has to be compressed into various formats, so it can be retrieved without delays.
generated by cgit v1.2.3 (git 2.46.0) at 2026-08-26 23:06:39 +0000