The 98th percentile position in the agebytes map was incorrectly
calculated: it assumed the transactions in the mempool all have unique
timestamps at second-granularity. This commit fixes this by correctly
finding the right cumulative number of transactions in the map suffix.
This bug could lead to an out-of-bounds write in the rare case that
all transactions in the mempool were received (and added to the mempool)
at a rate of at least 50 transactions per second. (More specifically,
the number of *unique* receive_time values, which have second-
granularity, must be at most 2% of the number of transactions in the
mempool for this crash to trigger.) If this condition is satisfied, 'it'
points to *before* the agebytes map, 'delta' gets a nonsense value, and
the value of 'i' in the first stats.histo-filling loop will be out of
bounds of stats.histo.
According to [1], std::random_shuffle is deprecated in C++14 and removed
in C++17. Since std::shuffle is available since C++11 as a replacement
and monero already requires C++11, this is a good replacement.
A cryptographically secure random number generator is used in all cases
to prevent people from perhaps copying an insecure std::shuffle call
over to a place where a secure one would be warranted. A form of
defense-in-depth.
[1]: https://en.cppreference.com/w/cpp/algorithm/random_shuffle
0605406 daemon: sort alt chains by height (moneromooo-monero)
4228ee0 daemon: add optional arguments to alt_chain_info (moneromooo-monero)
880ebfd daemon: add more chain specific info in alt_chain_info (moneromooo-monero)
35da33be blockchain: do not try to pop blocks down to the genesis block (moneromooo-monero)
4b51f9a3 core: do not commit half constructed batch db txn (moneromooo-monero)
d009f6dd rpc: fix get_block_hashes.bin from wallet on pruned blockchain (moneromooo-monero)
bb0ef5b1 blockchain: lock the blockchain while pruning (moneromooo-monero)
e9fac29a unit_tests/long_term_block_weight: some tweaks that seem to make more sense (stoffu)
467f4c7e tests/block_weight: use integer division when computing median (stoffu)
815d08dc tests/block_weight: remove unused MULTIPLIER_SMALL (stoffu)
661f1fb8 blockchain: remove unused calc of short_term_constraint (stoffu)
The db txn in add_block ending caused the entire overarching
batch txn to stop.
Also add a new guard class so a db txn can be stopped in the
face of exceptions.
Also use a read only db txn in init when the db itself is
read only, and do not save the max tx size in that case.
113e4877 blockchain_stats: fix sign in formatting function (moneromooo-monero)
adaea3ea various: remove unused variables (moneromooo-monero)
631ef00e blockchain: some debug info when adding txes-from-block fails (moneromooo-monero)
7c440915 Add get_tx_proof support, needed for new sanity check (cslashm)
98fdcb2a Add support for V11 protocol with BulletProofV2 and short amount. New scheme key destination contrfol Fix dummy decryption in debug mode (cslashm)
3a981a33 Add application version compatibility check. (cslashm)
ae6885f6 blockchain: incremental long term block weight cache (moneromooo-monero)
9b687c78 blockchain: simple cache for the long term block weights (moneromooo-monero)
The original intent of one false positive a week on average
was not met, since what we really want is not the probability
of having N blocks in T seconds, but either N blocks of fewer
in T seconds, or N blocks or more in T seconds.
Some of this could be cached since it calculates the same fairly
complex floating point values, but it seems pretty fast already.
0de14396 tests: add a CNv4 JIT test (moneromooo-monero)
24d281c3 crypto: plug CNv4 JIT into cn_slow_hash (moneromooo-monero)
78ab59ea crypto: clear cache after generating random program (moneromooo-monero)
b9a61884 performance_tests: add tests for new Cryptonight variants (moneromooo-monero)
fff23bf7 CNv4 JIT compiler for x86-64 and tests (SChernykh)
3dde67d8 blockchain: add v10 fork heights (moneromooo-monero)
2dbc487e Add support for V10 protocol with BulletProofV2 and short amount. (cslashm)
63cc02c0 Fix dummy decryption in debug mode (cslashm)
f0e55ceb fix log namespace (cslashm)
460da140 New scheme key destination contrfol (cslashm)
The 10 minute one will never trigger for 0 blocks, as it's still
fairly likely to happen even without the actual hash rate changing
much, so we add a 20 minute window, where it will (for 0 blocks)
and a one hour window.
This curbs runaway growth while still allowing substantial
spikes in block weight
Original specification from ArticMine:
here is the scaling proposal
Define: LongTermBlockWeight
Before fork:
LongTermBlockWeight = BlockWeight
At or after fork:
LongTermBlockWeight = min(BlockWeight, 1.4*LongTermEffectiveMedianBlockWeight)
Note: To avoid possible consensus issues over rounding the LongTermBlockWeight for a given block should be calculated to the nearest byte, and stored as a integer in the block itself. The stored LongTermBlockWeight is then used for future calculations of the LongTermEffectiveMedianBlockWeight and not recalculated each time.
Define: LongTermEffectiveMedianBlockWeight
LongTermEffectiveMedianBlockWeight = max(300000, MedianOverPrevious100000Blocks(LongTermBlockWeight))
Change Definition of EffectiveMedianBlockWeight
From (current definition)
EffectiveMedianBlockWeight = max(300000, MedianOverPrevious100Blocks(BlockWeight))
To (proposed definition)
EffectiveMedianBlockWeight = min(max(300000, MedianOverPrevious100Blocks(BlockWeight)), 50*LongTermEffectiveMedianBlockWeight)
Notes:
1) There are no other changes to the existing penalty formula, median calculation, fees etc.
2) There is the requirement to store the LongTermBlockWeight of a block unencrypted in the block itself. This is to avoid possible consensus issues over rounding and also to prevent the calculations from becoming unwieldy as we move away from the fork.
3) When the EffectiveMedianBlockWeight cap is reached it is still possible to mine blocks up to 2x the EffectiveMedianBlockWeight by paying the corresponding penalty.
Note: the long term block weight is stored in the database, but not in the actual block itself,
since it requires recalculating anyway for verification.
This will trigger if a reorg is seen. This may be used to do things
like stop automated withdrawals on large reorgs.
%s is replaced by the height at the split point
%h is replaced by the height of the new chain
%n is replaced by the number of new blocks after the reorg
b6534c40 ringct: remove unused senderPk from ecdhTuple (moneromooo-monero)
7d375981 ringct: the commitment mask is now deterministic (moneromooo-monero)
99d946e6 ringct: encode 8 byte amount, saving 24 bytes per output (moneromooo-monero)
cdc3ccec ringct: save 3 bytes on bulletproof size (moneromooo-monero)
f931e16c add a bulletproof version, new bulletproof type, and rct config (moneromooo-monero)
The blockchain prunes seven eighths of prunable tx data.
This saves about two thirds of the blockchain size, while
keeping the node useful as a sync source for an eighth
of the blockchain.
No other data is currently pruned.
There are three ways to prune a blockchain:
- run monerod with --prune-blockchain
- run "prune_blockchain" in the monerod console
- run the monero-blockchain-prune utility
The first two will prune in place. Due to how LMDB works, this
will not reduce the blockchain size on disk. Instead, it will
mark parts of the file as free, so that future data will use
that free space, causing the file to not grow until free space
grows scarce.
The third way will create a second database, a pruned copy of
the original one. Since this is a new file, this one will be
smaller than the original one.
Once the database is pruned, it will stay pruned as it syncs.
That is, there is no need to use --prune-blockchain again, etc.
c6d38718 core: include a dummy encrypted payment id when no payment is used (moneromooo-monero)
b7441c4a core, wallet: remember original text version of destination address (moneromooo-monero)
a9b1c04a crptonote_core: do not error out sending unparsable extra field (moneromooo-monero)
5ee6f037 blockchain: fix wrong hf version when popping multiple blocks (moneromooo-monero)
634d359a blockchain: use the version passed as parameter, not a new one (moneromooo-monero)
94a375d5 hardfork: remove batch transactions setup (moneromooo-monero)
6644b9b blockchain_db: remove a couple unused functions (moneromooo-monero)
ce594f5 blockchain_db: allocate known size vector only once (moneromooo-monero)
8332698 db_lmdb: inline check_open, it's trivial and called everywhere (moneromooo-monero)
5511563 db_lmdb: avoid pointless division (moneromooo-monero)
d1efe3d cryptonote: set tx hash on newly parsed txes when known (moneromooo-monero)
9cc68a2 tx_pool: add a few std::move where it can make a difference (moneromooo-monero)
While the lookups are faster, the zeroCommit calls have to be
done again when storing the new outputs in the db, which ends
up making the whole thing slower after all, and the ways this
can be cached aren't very nice code wise, so let's forget it
since the gains aren't very large anyway.
aee7a4e3 wallet_rpc_server: do not use RPC data if the call failed (moneromooo-monero)
1a0733e5 windows_service: fix memory leak (moneromooo-monero)
0dac3c64 unit_tests: do not rethrow a copy of an exception (moneromooo-monero)
5d9915ab cryptonote: fix get_unit for non default settings (moneromooo-monero)
d4f50cb1 remove some unused code (moneromooo-monero)
61163971 a few minor (but easy) performance tweaks (moneromooo-monero)
30023074 tests: slow_memmem now returns size_t (moneromooo-monero)
This avoids the miner erroring out trying to submit blocks
to a core that's already shut down (and avoids pegging
the CPU while we're busy shutting down).
Some of the inputs for block in a span will be from other earlier
blocks in that span. Keep track of those outputs so we don't have
to look them up again after those early blocks are added to the
blockchain.
5808530f blockchain: remove unused output_scan_worker parameter (moneromooo-monero)
1426209a blockchain: don't run threads if we have just one function to run (moneromooo-monero)
6f7a5fd4 db_lmdb: slight speedup getting array data from the blockchain (moneromooo-monero)
99fbe100 db_lmdb: save some string copies for readonly db keys/values (moneromooo-monero)
bf31447e tx_pool: speed up take_tx for transactions from blocks (moneromooo-monero)
4f005a77 tx_pool: remove unnecessary get_transaction_hash (moneromooo-monero)
593ef598 perf_timer: call reserve on new timer array (moneromooo-monero)
6ecc99ad core: avoid unnecessary tx/blob conversions (moneromooo-monero)
00cc1a16 unit_tests: notify test special case for the usual weirdo (moneromooo-monero)
73e504c1 rpc: adjust ring size error message now that too high is also possible (moneromooo-monero)
a5ca7f4f core: fix unmixable special case allowing ring size below 11 (moneromooo-monero)
To help protect one's privacy from traffic volume analysis
for people using Tor or I2P. This will really fly once we
relay txes on a timer rather than on demand, though.
Off by default for now since it's wasteful and doesn't bring
anything until I2P's in.
This happens for every historical tx when syncing, and the
unnecessary parsing is actually showing up on profile.
Since these are kept cached for just one block, this does
not increase memory usage after syncing.
This removes some small amount of fingerprinting entropy.
There is no consensus rule to require this since this field
is technically free form, and a transaction is free to have
custom data in it.
b2bb9312 blockchain: simplify output distribution code (moneromooo-monero)
befdcbf4 db_lmdb: do not use base for cumulative distribution (moneromooo-monero)
This avoids constant rechecking of the same things each time
a miner asks for the block template. The tx pool maintains
a cookie to allow users to detect when the pool state changed,
which means the block template needs rebuilding.
149da42 db_lmdb: enable batch transactions by default (stoffu)
34cb6b4 add --regtest and --fixed-difficulty for regression testing (vicsn)
9e1403e update get_info RPC and bump RPC version (vicsn)
207b66e first new functional tests (vicsn)
This gets rid of the temporary precalc cache.
Also make the RPC able to send data back in binary or JSON,
since there can be a lot of data
This bumps the LMDB database format to v3, with migration.
