1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
|
//! The Tor directory mirror implementation.
//!
//! # Specifications
//!
//! * [Directory cache operation](https://spec.torproject.org/dir-spec/directory-cache-operation.html).
//!
//! # Rationale
//!
//! The network documents specified in the directory specification form a
//! fundamental part within the Tor protocol, namely the creation and distribution
//! of a canonical list, listing all relays present in the Tor network, thereby
//! giving all clients a unified view of the entire Tor network, a fact that
//! is very important for defending against partitioning attacks and other potential
//! attacks in the domain of distributed networks.
//!
//! These network documents are generated, signed, and served by so called
//! "directory authorities", a set of 10-ish highly trusted Tor relays more or
//! less governing the entirety of the Tor network.
//!
//! Now here comes the bottleneck: Tor has millions of active daily users but
//! only 10-ish relays responsible for these crucial documents. Having all
//! clients download from those 10-ish relays would present an immense overload
//! to those, thereby potentially shutting the entire Tor network down, if the
//! amount of traffic to those relays is so high, that they are unable to
//! communicate and coordinate under themselves.
//!
//! Fortunately, all network documents are either directly or indirectly signed
//! by well-known keys of directory authorities, thereby making mirroring them
//! trivially possible, due the fact that authenticity can be established outside
//! the raw TLS connection thanks to cryptographic signatures.
//!
//! This is the place where directory mirrors come in hnady. Directory mirrors
//! (previously known as "directory caches") are ordinary relays that mirror all
//! network documents from the authorities, by implementing the respective routes
//! for all HTTP GET endpoints from the relays.
//!
//! The network documents are usually served through ordinary Tor circuits,
//! by accepting incoming connections through `RELAY_BEGIN_DIR` cells.
//! In the past, this was done by some relays optionally enabling an additional
//! socket on the ordinary Internet through a dedicated SocketAddr, known as
//! "directory address". Since about 2020, this is no longer done. However,
//! the functionality continues to persist and this module is written fairly
//! agnostic on how it accepts such connections, as directory authorities continue
//! to advertise their directory address.
use std::{convert::Infallible, path::PathBuf};
use futures::Stream;
use tokio::io::{AsyncRead, AsyncWrite};
use tor_dircommon::{
authority::AuthorityContacts,
config::{DirTolerance, DownloadScheduleConfig},
};
mod operation;
/// Core data type of a directory mirror.
///
/// # External Notes
///
/// This structure serves as the entrence point to the [`mirror`](crate::mirror)
/// API. It represents an instance that is launchable using [`DirMirror::serve`].
/// Calling this method consumes the instance, as this is the common behavior
/// for objects representing server-like things, in order to not imply that this
/// instance serves as a mere configuration template only.
///
/// # Internal Notes
///
/// For now, this data structure only holds configuration options as an ad-hoc
/// replacement for a yet missing hypothetical `DirMirrorConfig` structure.
///
/// I assume that in the future, regardless of the configuration, this might also
/// hold other fields such as access to the database pool, etc. The question
/// is whether this structure will be passed around with locking mechanisms
/// or will just be used as a way to extract configuration options initially
/// in the consuming function, which then applies further wrapping or not.
#[derive(Debug)]
#[non_exhaustive]
pub struct DirMirror {
/// The [`PathBuf`] where the [`database`](crate::database) is located.
path: PathBuf,
/// The [`AuthorityContacts`] data structure for contacting authorities.
authorities: AuthorityContacts,
/// The [`DownloadScheduleConfig`] used for properly retrying downloads.
schedule: DownloadScheduleConfig,
/// The [`DirTolerance`] to tolerate clock skews.
tolerance: DirTolerance,
}
impl DirMirror {
/// Creates a new [`DirMirror`] with a given set of configuration options.
///
/// # Parameters
///
/// * `path`: The [`PathBuf`] where the database is located.
/// * `authorities`: The [`AuthorityContacts`] data structure for contacting authorities.
/// * `schedule`: The [`DownloadScheduleConfig`] used for properly retrying downloads.
/// * `tolerance`: The [`DirTolerance`] to tolerate clock skews.
///
/// # Notes
///
/// **Beware of [`DirTolerance::default()`]!**, as the default values are
/// intended for clients, not directory mirrors. Tolerances of several days
/// are not recommended for directory mirrors. Consider using something in
/// the minute range instead, such as `60s`, which is what ctor uses.[^1]
///
/// TODO DIRMIRROR: This is unacceptable for the actual release. We **NEED**
/// a proper way to configure this, such as with a `DirMirrorConfig` struct
/// that can properly serialize from configuration files and such. However,
/// this task is not a trivial one and maybe one of the hardest parts of this
/// entire development, as it would involve a radical change to many higher
/// level crates. The reason for this being, that we need a clean way to
/// share "global" settings such as the list of authorities into various
/// sub-configurations, such as the configuration for the directory mirror.
/// We must not offer different configurations for the list of authorities
/// for those different components, that would result in lots of boilerplate
/// and potentially wrong execution given that those resources are affecting
/// so many parts of the Tor protocol that a consistent view must be assumed
/// in order to avoid surprising behavior.
///
/// [^1]: <https://gitlab.torproject.org/tpo/core/tor/-/blob/0b20710/src/feature/nodelist/networkstatus.c#L1890>.
pub fn new(
path: PathBuf,
authorities: AuthorityContacts,
schedule: DownloadScheduleConfig,
tolerance: DirTolerance,
) -> Self {
Self {
path,
authorities,
schedule,
tolerance,
}
}
/// Consumes the [`DirMirror`] by running endlessly in the current task.
///
/// This method accepts a `listener`, which is a [`Stream`] yielding a
/// [`Result`] in order to model a generic way of accepting incoming
/// connections. Think of `S` as the file descriptor you would call
/// `accept(2)` upon if you were in C. The idea behind this generic is,
/// as outlined in the module documentation, that a [`DirMirror`] can
/// handle incoming connections in multiple ways, such as by serving
/// through an ordinary TCP socket or through a Tor circuit in combination
/// with a `RELAY_BEGIN_DIR` cell. How this is concretely done, is outside
/// the scope of this crate; instead we provide the primitives making such
/// flexibility possible.
///
// TODO DIRMIRROR: can we change the listener to be a
// Stream<Item = T> + Unpin instead of Stream<Item = Result<T, E>> + Unpin?
// We expect the calling code to filter out any errors before handing
// the stream over to DirMirror::serve().
//
// See https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/4222#note_3437135
#[allow(clippy::unused_async)] // TODO
pub async fn serve<S, T, E>(self, mut listener: S) -> Result<(), Infallible>
where
S: Stream<Item = Result<T, E>> + Unpin,
T: AsyncRead + AsyncWrite + Unpin + Send + 'static,
E: std::error::Error,
{
use futures::StreamExt as _;
use tokio::io::AsyncWriteExt as _;
// TODO DIRMIRROR: Replace this with the real implementation
while let Some(s) = listener.next().await {
let mut s = match s {
Ok(s) => s,
Err(e) => {
tracing::debug!("{e:?}");
continue;
}
};
let serve_dummy_response = async {
s.write_all(b"HTTP/1.1 500 Internal Server Error\r\n\r\n")
.await?;
s.flush().await?;
Ok::<(), std::io::Error>(())
};
if let Err(e) = serve_dummy_response.await {
tracing::debug!(e=?e, "failed to serve dirmirror connection");
}
}
Ok(())
}
}
|
