Commit Graph

20863 Commits

Author SHA1 Message Date
David Goulet ae4e5b9824 token: Fix uint32_t to uint64_t conversion
Unfortunately, the units passed to
monotime_coarse_stamp_units_to_approx_msec() was always 0 due to a type
conversion.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-16 15:05:41 -04:00
Nick Mathewson c7d3de216c Merge branch 'maint-0.3.3' 2018-04-16 13:48:39 -04:00
Nick Mathewson 9ef4c05df8 Merge branch 'maint-0.3.2' into maint-0.3.3 2018-04-16 13:48:23 -04:00
Nick Mathewson c5899d5cf3 Merge branch 'maint-0.3.1' into maint-0.3.2 2018-04-16 13:48:23 -04:00
Nick Mathewson 5e0fbd7006 Merge branch 'maint-0.2.9' into maint-0.3.1 2018-04-16 13:48:23 -04:00
Nick Mathewson 0e13ff4815 Fix an LCOV exclusion pattern in address.c 2018-04-16 13:48:21 -04:00
Nick Mathewson 3463b4e065 Merge branch 'maint-0.3.3' 2018-04-16 10:06:21 -04:00
Nick Mathewson 22845df2a7 Merge remote-tracking branch 'dgoulet/bug25226_033_02' into maint-0.3.3 2018-04-16 10:04:36 -04:00
David Goulet d064122e70 relay: Implement a circuit cell queue maximum size
This commit introduces the consensus parameter "circ_max_cell_queue_size"
which controls the maximum number of cells a circuit queue should have.

The default value is currently 50000 cells which is above what should be
expected but keeps us a margin of error for padding cells.

Related to this is #9072. Back in 0.2.4.14-alpha, we've removed that limit due
to a Guard discovery attack. Ticket #25226 details why we are putting back the
limit due to the memory pressure issue on relays.

Fixes #25226

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-16 09:59:12 -04:00
Nick Mathewson 197d1992db Remove old tor-fw-helper README from EXTRA_DIST
We removed this file, but didn't take it out of EXTRA_DIST -- thus
breaking "make dist".
2018-04-16 09:52:15 -04:00
Nick Mathewson 3ee4c9b1fa bump to 0.3.3.5-rc-dev 2018-04-15 15:41:03 -04:00
Nick Mathewson b65024f57d bump to 0.3.3.5-rc 2018-04-14 12:21:36 -04:00
Nick Mathewson 4b58b97c68 32-bit compilation warnings 2018-04-13 17:01:03 -04:00
David Goulet c2f83746f4 token_bucket: Fix indentation
Both header and code file had some indentation issues after mass renaming.

No code behavior change.

Signed-off-by: David Goulet <dgoulet@torproject.org>
2018-04-13 16:58:49 -04:00
Nick Mathewson 34c2574aa9 Merge branch 'token_bucket_once_again_squashed' 2018-04-13 16:31:58 -04:00
Nick Mathewson 1b31195b4f Fix "make check-spaces" 2018-04-13 16:31:47 -04:00
Nick Mathewson 003e6595bf Refactor "timestamp" not to be its own type coupled to token buffers
Really, the uint32_t is only an optimization; any kind of unit
should work fine.  Some users might want to use time_t or
monotime_coarse_t or something like that.
2018-04-13 16:31:47 -04:00
Nick Mathewson 2307bef7a2 Move token_bucket_raw_* functions to the start of the module.
(These functions were previously helper functions for
token_bucket_rw_t).
2018-04-13 16:31:47 -04:00
Nick Mathewson 9c405ba595 Never pick a rate of 0.
(The tests caught this one.)
2018-04-13 16:31:47 -04:00
Nick Mathewson 0b40ed5e70 Start re-refactoring the token bucket interface.
Begin by creating a lowest-level triple of the types needed to
implement a token bucket: a configuration, a timestamp, and the raw
bucket itself.

Note that for low-level buckets, the units of the timestamp and the
bucket itself are unspecified: each user can use a different type.

(This patch breaks check-spaces; a later patch will fix it)
2018-04-13 16:31:47 -04:00
Nick Mathewson c9de30c590 Merge branch 'maint-0.3.3' 2018-04-13 13:05:53 -04:00
Nick Mathewson 61d87dfa15 Merge branch 'postloop_callbacks_2' 2018-04-13 12:12:46 -04:00
Nick Mathewson 4c03af4880 Remove tell_event_loop_to_run_external_code() per review
(This function is no longer used.)
2018-04-13 12:11:22 -04:00
Nick Mathewson 03b96882de Rename token_bucket_t to token_bucket_rw_t.
This is a simple search-and-replace to rename the token bucket type
to indicate that it contains both a read and a write bucket, bundled
with their configuration.  It's preliminary to refactoring the
bucket type.
2018-04-13 10:54:26 -04:00
Nick Mathewson b152d62cee Merge branch 'token_bucket_refactor_squashed' 2018-04-13 10:47:24 -04:00
Nick Mathewson 62f4d5a265 Add a unit test for post-loop events
This test works by having two post-loop events activate one another
in a tight loop.  If the "post-loop" mechanism didn't work, this
would be enough to starve all other events.
2018-04-13 10:44:15 -04:00
Nick Mathewson 320bd2b3a5 Move connection_ap_attach_pending(0) into a postloop event
This is a second motivating case for our postloop event logic.
2018-04-13 10:44:15 -04:00
Nick Mathewson 5719dfb48f Move the "activate linked connections" logic to a postloop event.
A linked connection_t is one that gets its I/O, not from the
network, but from another connection_t.  When such a connection has
something to write, we want the corresponding connection to run its
read callback ... but not immediately, to avoid infinite recursion
and/or event loop starvation.

Previously we handled this case by activating the read events
outside the event loop.  Now we use the "postloop event" logic.
This lets us simplify do_main_loop_once() a little.
2018-04-13 10:44:15 -04:00
Nick Mathewson c5a3e2ca44 Generic mechaism for "post-loop" callbacks
We've been labeling some events as happening "outside the event
loop", to avoid Libevent starvation.  This patch provides a cleaner
mechanism to avoid that starvation.

For background, the problem here is that Libevent only scans for new
events once it has run all its active callbacks.  So if the
callbacks keep activating new callbacks, they could potentially
starve Libevent indefinitely and keep it from ever checking for
timed, socket, or signal events.

To solve this, we add the ability to label some events as
"post-loop".  The rule for a "post-loop" event is that any events
_it_ activates can only be run after libevent has re-scanned for new
events at least once.
2018-04-13 10:44:15 -04:00
Nick Mathewson ad57b1279a Disable load_geoip_file() tests on windows
See bug #25787 for discussion; we should have a better fix here.
2018-04-13 10:42:19 -04:00
Nick Mathewson 787bafc0f9 Increase tolerances for imprecise time. 2018-04-13 10:41:15 -04:00
Nick Mathewson 3f514fe3b1 Accept small hops backward in the monotonic timer. 2018-04-13 10:41:15 -04:00
Nick Mathewson 12f58f2f87 Remove a bunch of int casts; make clang happier. 2018-04-13 10:41:14 -04:00
Nick Mathewson 6be994fa71 Ensure that global buckets are updated on configuration change 2018-04-13 10:41:14 -04:00
Nick Mathewson a38fd9bc5b Replace the global buckets with token_bucket_t 2018-04-13 10:41:14 -04:00
Nick Mathewson 9fced56ef1 Refactor or_connection token buckets to use token_bucket_t 2018-04-13 10:41:14 -04:00
Nick Mathewson 8a85239746 Add a helper function to decrement read and write at the same time 2018-04-13 10:41:14 -04:00
Nick Mathewson c376200f6a Add a new token-bucket backend abstraction, with tests
This differs from our previous token bucket abstraction in a few
ways:

  1) It is an abstraction, and not a collection of fields.
  2) It is meant to be used with monotonic timestamps, which should
     produce better results than calling gettimeofday over and over.
2018-04-13 10:41:14 -04:00
Nick Mathewson d8ef9a2d1e Expose a function that computes stamp units from msec.
(It turns out we can't just expose STAMP_TICKS_PER_SECOND, since
Apple doesn't have that.)
2018-04-13 10:41:08 -04:00
Nick Mathewson 2d6914e391 Refine extend_info_for_node's "enough info" check once again.
In d1874b4339, we adjusted this check so that we insist on
using routerinfos for bridges.  That's almost correct... but if we
have a bridge that is also a regular relay, then we should use
insist on its routerinfo when connecting to it as a bridge
(directly), and be willing to use its microdescriptor when
connecting to it elsewhere in our circuits.

This bug is a likely cause of some (all?) of the (exit_ei == NULL)
failures we've been seeing.

Fixes bug 25691; bugfix on 0.3.3.4-alpha
2018-04-12 16:56:29 -04:00
Nick Mathewson d3b9b5a3dd Remove windows log_from_handle as unused.
This function was only used by PortForwardingHelper, which was
removed in 9df110cd72.  Its presence caused warnings on windows.
2018-04-12 12:38:46 -04:00
Nick Mathewson f0887e30dd Merge branch 'maint-0.3.3' 2018-04-12 12:31:41 -04:00
Nick Mathewson 46795a7be6 Attempt to fix 32-bit clang builds, which broke with 31508a0abc
When size_t is 32 bits, the unit tests can't fit anything more than
4GB-1 into a size_t.

Additionally, tt_int_op() uses "long" -- we need tt_u64_op() to
safely test uint64_t values for equality.

Bug caused by tests for #24782 fix; not in any released Tor.
2018-04-12 12:30:36 -04:00
Nick Mathewson 467c882baa Merge branch 'maint-0.3.3' 2018-04-12 12:25:51 -04:00
Nick Mathewson 4aaa4215e7 Attempt to fix 32-bit builds, which broke with 31508a0abc
When size_t is 32 bits, doing "size_t ram; if (ram > 8GB) { ... }"
produces a compile-time warning.

Bug caused by #24782 fix; not in any released Tor.
2018-04-12 12:25:09 -04:00
Nick Mathewson 037fb0c804 Merge branch 'maint-0.3.3' 2018-04-12 11:14:42 -04:00
Alexander Færøy 31508a0abc Use less memory for MaxMemInQueues for machines with more than 8 GB of RAM.
This patch changes the algorithm of compute_real_max_mem_in_queues() to
use 0.4 * RAM iff the system has more than or equal to 8 GB of RAM, but
will continue to use the old value of 0.75 * RAM if the system have less
than * GB of RAM available.

This patch also adds tests for compute_real_max_mem_in_queues().

See: https://bugs.torproject.org/24782
2018-04-12 11:14:16 -04:00
Alexander Færøy 5633a63379 Use STATIC for compute_real_max_mem_in_queues
This patch makes compute_real_max_mem_in_queues use the STATIC macro,
which allows us to test the function.

See: https://bugs.torproject.org/24782
2018-04-12 10:51:48 -04:00
Alexander Færøy bd42367a1e Make get_total_system_memory mockable.
This patch makes get_total_system_memory mockable, which allows us to
alter the return value of the function in tests.

See: https://bugs.torproject.org/24782
2018-04-12 10:51:45 -04:00
Nick Mathewson a51630cc9a Merge branch 'maint-0.3.3' 2018-04-11 15:38:00 -04:00