This curbs runaway growth while still allowing substantial
spikes in block weight
Original specification from ArticMine:
here is the scaling proposal
Define: LongTermBlockWeight
Before fork:
LongTermBlockWeight = BlockWeight
At or after fork:
LongTermBlockWeight = min(BlockWeight, 1.4*LongTermEffectiveMedianBlockWeight)
Note: To avoid possible consensus issues over rounding the LongTermBlockWeight for a given block should be calculated to the nearest byte, and stored as a integer in the block itself. The stored LongTermBlockWeight is then used for future calculations of the LongTermEffectiveMedianBlockWeight and not recalculated each time.
Define: LongTermEffectiveMedianBlockWeight
LongTermEffectiveMedianBlockWeight = max(300000, MedianOverPrevious100000Blocks(LongTermBlockWeight))
Change Definition of EffectiveMedianBlockWeight
From (current definition)
EffectiveMedianBlockWeight = max(300000, MedianOverPrevious100Blocks(BlockWeight))
To (proposed definition)
EffectiveMedianBlockWeight = min(max(300000, MedianOverPrevious100Blocks(BlockWeight)), 50*LongTermEffectiveMedianBlockWeight)
Notes:
1) There are no other changes to the existing penalty formula, median calculation, fees etc.
2) There is the requirement to store the LongTermBlockWeight of a block unencrypted in the block itself. This is to avoid possible consensus issues over rounding and also to prevent the calculations from becoming unwieldy as we move away from the fork.
3) When the EffectiveMedianBlockWeight cap is reached it is still possible to mine blocks up to 2x the EffectiveMedianBlockWeight by paying the corresponding penalty.
Note: the long term block weight is stored in the database, but not in the actual block itself,
since it requires recalculating anyway for verification.
When all our outgoing peer slots are filled, we cycle one peer at
a time looking for syncing peers until we have at least two such
peers. This brings two advantages:
- Peers without incoming connections will find more syncing peers
that before, thereby strengthening network decentralization
- Peers will have more resistance to isolation attacks, as they
are more likely to find a "good" peer than they were before
NetBSD emits:
warning: Warning: reference to the libc supplied alloca(3); this most likely will not work. Please use the compiler provided version of alloca(3), by supplying the appropriate compiler flags (e.g. not -std=c89).
and man 3 alloca says:
Normally, gcc(1) translates calls to alloca() with inlined code. This is not done when either the -ansi, -std=c89, -std=c99, or the
-std=c11 option is given and the header <alloca.h> is not included. Otherwise, (without an -ansi or -std=c* option) the glibc version of
<stdlib.h> includes <alloca.h> and that contains the lines:
#ifdef __GNUC__
#define alloca(size) __builtin_alloca (size)
#endif
It looks like alloca is a bad idea in modern C/C++, so we use
VLAs for C and std::vector for C++.
Building with docker is arguably easier and more familiar to most people
than either kvm, or lxc.
This commit also relaxes the back compat requirement a bit. 32 bit linux
now uses glibc version 2.0. Also, the docker shell could not handle gcc arguments
containing spaces, so the explicit '-DFELT_TYPE' declaration was dropped.
Lastly, this removes some packages from the osx descriptor.
```
Undefined symbols for architecture x86_64:
"common_category()", referenced from:
make_error_code(common_error) in parse.cpp.o
make_error_code(common_error) in tor_address.cpp.o
"boost::system::detail::system_category_ncx()", referenced from:
boost::system::system_category() in parse.cpp.o
boost::system::system_category() in socks.cpp.o
boost::system::system_category() in libepee.a(net_utils_base.cpp.o)
"boost::system::detail::generic_category_ncx()", referenced from:
boost::system::generic_category() in parse.cpp.o
boost::system::generic_category() in socks.cpp.o
boost::system::generic_category() in tor_address.cpp.o
boost::system::generic_category() in libepee.a(string_tools.cpp.o)
boost::system::generic_category() in libepee.a(net_utils_base.cpp.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [src/net/libnet.dylib] Error 1
make[2]: *** [src/net/CMakeFiles/net.dir/all] Error 2
```
The change made for v2 broke v1, and we have no way to know which
version we're serializing here. However, since we don't actually
care about space savings in this case, we continue serialiazing
both mask and amount.
RPC connections now have optional tranparent SSL.
An optional private key and certificate file can be passed,
using the --{rpc,daemon}-ssl-private-key and
--{rpc,daemon}-ssl-certificate options. Those have as
argument a path to a PEM format private private key and
certificate, respectively.
If not given, a temporary self signed certificate will be used.
SSL can be enabled or disabled using --{rpc}-ssl, which
accepts autodetect (default), disabled or enabled.
Access can be restricted to particular certificates using the
--rpc-ssl-allowed-certificates, which takes a list of
paths to PEM encoded certificates. This can allow a wallet to
connect to only the daemon they think they're connected to,
by forcing SSL and listing the paths to the known good
certificates.
To generate long term certificates:
openssl genrsa -out /tmp/KEY 4096
openssl req -new -key /tmp/KEY -out /tmp/REQ
openssl x509 -req -days 999999 -sha256 -in /tmp/REQ -signkey /tmp/KEY -out /tmp/CERT
/tmp/KEY is the private key, and /tmp/CERT is the certificate,
both in PEM format. /tmp/REQ can be removed. Adjust the last
command to set expiration date, etc, as needed. It doesn't
make a whole lot of sense for monero anyway, since most servers
will run with one time temporary self signed certificates anyway.
SSL support is transparent, so all communication is done on the
existing ports, with SSL autodetection. This means you can start
using an SSL daemon now, but you should not enforce SSL yet or
nothing will talk to you.
According to [1], the ios_base::streampos member type is deprecated, and
removed in C++17. This type was an alias for pos_type, which this commit
uses instead.
[1]: https://en.cppreference.com/w/cpp/io/ios_base
- Support for ".onion" in --add-exclusive-node and --add-peer
- Add --anonymizing-proxy for outbound Tor connections
- Add --anonymous-inbounds for inbound Tor connections
- Support for sharing ".onion" addresses over Tor connections
- Support for broadcasting transactions received over RPC exclusively
over Tor (else broadcast over public IP when Tor not enabled).
23813c71 blockchain: add --reorg-notify (moneromooo-monero)
f6db59b0 notify: handle arbitrary tags (moneromooo-monero)
ff959216 notify: warn if the spec contains one of '"\ (moneromooo-monero)
13852678 common: set MONERO_DEFAULT_LOG_CATEGORY for notify and spawn (moneromooo-monero)
This will trigger if a reorg is seen. This may be used to do things
like stop automated withdrawals on large reorgs.
%s is replaced by the height at the split point
%h is replaced by the height of the new chain
%n is replaced by the number of new blocks after the reorg
b6534c40 ringct: remove unused senderPk from ecdhTuple (moneromooo-monero)
7d375981 ringct: the commitment mask is now deterministic (moneromooo-monero)
99d946e6 ringct: encode 8 byte amount, saving 24 bytes per output (moneromooo-monero)
cdc3ccec ringct: save 3 bytes on bulletproof size (moneromooo-monero)
f931e16c add a bulletproof version, new bulletproof type, and rct config (moneromooo-monero)
The blockchain prunes seven eighths of prunable tx data.
This saves about two thirds of the blockchain size, while
keeping the node useful as a sync source for an eighth
of the blockchain.
No other data is currently pruned.
There are three ways to prune a blockchain:
- run monerod with --prune-blockchain
- run "prune_blockchain" in the monerod console
- run the monero-blockchain-prune utility
The first two will prune in place. Due to how LMDB works, this
will not reduce the blockchain size on disk. Instead, it will
mark parts of the file as free, so that future data will use
that free space, causing the file to not grow until free space
grows scarce.
The third way will create a second database, a pruned copy of
the original one. Since this is a new file, this one will be
smaller than the original one.
Once the database is pruned, it will stay pruned as it syncs.
That is, there is no need to use --prune-blockchain again, etc.
c6d38718 core: include a dummy encrypted payment id when no payment is used (moneromooo-monero)
b7441c4a core, wallet: remember original text version of destination address (moneromooo-monero)
a9b1c04a crptonote_core: do not error out sending unparsable extra field (moneromooo-monero)
464097e5 blockchain_ancestry: allow getting ancestry of a single output (moneromooo-monero)
a6216d1a blockchain_db: allow getting output keys without commitment (moneromooo-monero)
15904610 simplewallet: remove extra colon in a few calls to input_secure_line (stoffu)
247dab73 simplewallet: avoid conversion to string in input_secure_line (stoffu)
bf9ef7ad simplewallet: factor yesno hint into input_line (stoffu)
This involved a reorg of the code, to factor and speedup some bits,
as well as using the cache for all modes, and making both modes
usable in the same run.
5ee6f037 blockchain: fix wrong hf version when popping multiple blocks (moneromooo-monero)
634d359a blockchain: use the version passed as parameter, not a new one (moneromooo-monero)
94a375d5 hardfork: remove batch transactions setup (moneromooo-monero)
Half of the patch was correct, but half was introducing another bug,
where a wallet asking for a fork that the daemon does not know about
yet would decide to use those rules.
(1) If the user denies something on the Ledger,
a proper error message is now shown.
(2) Ledger doesn't time out anymore while waiting
on user input.
(3) Lower the timeout to 2 seconds, this is enough for
normal Ledger <-> System communication.
This avoids the constant message about needed to run refresh
to enter a password.
Also mention the txpool when asking for the password if the
reason is a pool tx.
get_output_key method is commonly used when working with txs and their key images. Because the method is not const, passing blockchain object though const& or pointers to const is not possible in this context. This is especially problematic in external projects (e.g., projects in moneroexamples) that use monero C++ api to operate on the blockchain and txs.
Thus, having get_output_key method will simplify moving blockchain object around through const references and pointers to const objects.
93c59b29 perf_timer: check allowed categories before logging (moneromooo-monero)
6a507dab perf_timer: add a way to get and reset the current time (moneromooo-monero)
c1581a5b perf_timer: only log to file (moneromooo-monero)
a13eb0a1 epee: speed up string matching a bit (moneromooo-monero)
3a3858dc epee: avoid string allocation when parsing a pod from string (moneromooo-monero)
6644b9b blockchain_db: remove a couple unused functions (moneromooo-monero)
ce594f5 blockchain_db: allocate known size vector only once (moneromooo-monero)
8332698 db_lmdb: inline check_open, it's trivial and called everywhere (moneromooo-monero)
5511563 db_lmdb: avoid pointless division (moneromooo-monero)
d1efe3d cryptonote: set tx hash on newly parsed txes when known (moneromooo-monero)
9cc68a2 tx_pool: add a few std::move where it can make a difference (moneromooo-monero)
While the lookups are faster, the zeroCommit calls have to be
done again when storing the new outputs in the db, which ends
up making the whole thing slower after all, and the ways this
can be cached aren't very nice code wise, so let's forget it
since the gains aren't very large anyway.
Help messages describe the commands usage. When users run the command
with wrong arguments, it usually helpfully offers the usage, too.
Unfortunately, these two usage messages were duplicated in the code and
started to get out of sync.
Fixing with constant strings.
- docker protobuf dependencies, cross-compilation
- device/trezor protobuf build fixes, try_compile
- libusb built under all platforms, used by trezor for direct connect
ab783b17 easylogging++: ensure logger is initialized before main (moneromooo-monero)
9b69a0ae daemon: print monero version at startup when calling a detached daemon (moneromooo-monero)
4d71d463 mlocker: remove early page size log (moneromooo-monero)
a48f2dab blockchain_prune_known_spent_data: blackball file is now optional (moneromooo-monero)
17b45725 Outputs where all amounts are known spent can now be pruned (moneromooo-monero)
- checks if the device is in the correct usable state
- implements check for the v2.0.9 firmware which does not support payment IDs
- simple transacttion check, payment id fmt consistency
- minor fixes, refactoring, webusb session counting fix
When doing a first refresh on HW-token based wallet KI sync is required if money were received. Received money may indicate wallet was already used before the restore I.e., some transaction could have been already sent from the wallet. The spent UTXO would not be detected as spent which could lead to double spending errors on submitting a new transaction.
Thus if the wallet is HW-token based with the cold signing protocol and the first refresh detected received money the user is asked to perform the key image sync.
- adds a new option `--hw-device-deriv-path` to the simple wallet. Enables to specify wallet derivation path / wallet code (path avoided so it can be misinterpreted as a file path).
- devices can use different derivation mechanisms. Trezor uses standard SLIP-10 mechanism with fixed SLIP-44 prefix for Monero
- Trezor: when empty, the default derivation mechanism is used with 44'/128'/0'. When entered the derivation path is 44'/128'/PATH.
- Trezor: the path is always taken as elements are hardened (1<<31 bit turned on)
aee7a4e3 wallet_rpc_server: do not use RPC data if the call failed (moneromooo-monero)
1a0733e5 windows_service: fix memory leak (moneromooo-monero)
0dac3c64 unit_tests: do not rethrow a copy of an exception (moneromooo-monero)
5d9915ab cryptonote: fix get_unit for non default settings (moneromooo-monero)
d4f50cb1 remove some unused code (moneromooo-monero)
61163971 a few minor (but easy) performance tweaks (moneromooo-monero)
30023074 tests: slow_memmem now returns size_t (moneromooo-monero)
This avoids the miner erroring out trying to submit blocks
to a core that's already shut down (and avoids pegging
the CPU while we're busy shutting down).
- simple device callback object added. Device can request passphrase/PIN entry via the callback or notify user some action is required
- callback is routed to wallet2, which routes the callback to i_wallet_callback so CLI or GUI wallets can support passphrase entry for HW tokens
- wallet: device open needs wallet callback first - passphrase protected device needs wallet callback so user can enter passphrase
Only for pre rct for obvious reasons.
Note: DO NOT use a known spent list which includes outputs
which are not known spent. If the list includes any output
that's just strongly thought to be spent, but not provably
so, you risk finding yourself unable to sync past the point
where that output is spent.
I estimate only 200 MB saved on current mainnet though,
unless the new blackballing rule unearths a good amount of
large-amount-set extra spent outs.
Some of the inputs for block in a span will be from other earlier
blocks in that span. Keep track of those outputs so we don't have
to look them up again after those early blocks are added to the
blockchain.
- webusb transport based on libusb added. Provides direct access to Trezor via USB, no need for Trezor bridge.
- trezor protocol message handler improved, no recursion used. Ready for upcoming integration tests.
- libusb (for docker) bumped from v1.0.9 to v1.0.22, newer version required for webusb transport, for device enumeration.
- cmake improvements and fixes. Cmake Trezor checks are moved to a dedicated CheckTrezor.cmake file. In case of a problem Trezor is excluded from build.
- ifdefs made consistent to Ledger.
- UDP Transport enumeration disabled by default in release mode
Motivated by https://monero.stackexchange.com/questions/10483
Some exchanges appear to have customized the wallet software
in an inappropriate way, making the tx extra field partially
unreadable. PR #3716 changed the wallet behavior disallowing
such partially valid tx extra.
An example tx reported by the user is
e87c675a85f34ecac58a8846613d25062f1813e1023c552b705afad32b972c38
where the normal tx pubkey appears again with the aditional
tx pubkeys tag `04` which is inappropriate.
03fc731b p2p: less frequent incoming connections check (moneromooo-monero)
14a5c206 p2p: tone down "no incoming connections" warning to info if in peers is 0 (moneromooo-monero)
This can go out of sync with m_core's nettype if you run in fakechain
mode since entering fakechain mode is done through code not the command
line and core_rpc_server only looks at the command line to figure out
the nettype.
5808530f blockchain: remove unused output_scan_worker parameter (moneromooo-monero)
1426209a blockchain: don't run threads if we have just one function to run (moneromooo-monero)
6f7a5fd4 db_lmdb: slight speedup getting array data from the blockchain (moneromooo-monero)
99fbe100 db_lmdb: save some string copies for readonly db keys/values (moneromooo-monero)
bf31447e tx_pool: speed up take_tx for transactions from blocks (moneromooo-monero)
4f005a77 tx_pool: remove unnecessary get_transaction_hash (moneromooo-monero)
593ef598 perf_timer: call reserve on new timer array (moneromooo-monero)
6ecc99ad core: avoid unnecessary tx/blob conversions (moneromooo-monero)
00cc1a16 unit_tests: notify test special case for the usual weirdo (moneromooo-monero)
6d3311ac libwallet_api_tests: add missing dependency on Boost Locale and ICU (stoffu)
f0459abb cmake: move Boost_LOCALE_LIBRARY out of ICU_LIBRARIES (stoffu)
73e504c1 rpc: adjust ring size error message now that too high is also possible (moneromooo-monero)
a5ca7f4f core: fix unmixable special case allowing ring size below 11 (moneromooo-monero)
To help protect one's privacy from traffic volume analysis
for people using Tor or I2P. This will really fly once we
relay txes on a timer rather than on demand, though.
Off by default for now since it's wasteful and doesn't bring
anything until I2P's in.
5f614ba9 simplewallet: print the number of show/all transfers (moneromooo-monero)
8d71b2b1 wallet2: only export necessary outputs and key images (moneromooo-monero)
769ae42a wallet2: faster output and key image import/export (moneromooo-monero)
Undefined symbols for architecture x86_64:
"cryptonote::core::get_output_distribution(unsigned long long, unsigned long long, unsigned long long, unsigned long long&, std::__1::vector<unsigned long long, std::__1::allocator<unsigned long long> >&, unsigned long long&) const", referenced from:
cryptonote::rpc::RpcHandler::get_output_distribution(cryptonote::core&, unsigned long long, unsigned long long, unsigned long long, bool) in rpc_handler.cpp.o
74fb3d88 multiexp: some minor speedups (moneromooo-monero)
a6d2e246 bulletproofs: only enable profiling on request (moneromooo-monero)
a110e6aa multiexp: tune which variants to use for which number of points (moneromooo-monero)
8b476722 bulletproofs: speedup prover (moneromooo-monero)
6f9ae5b6 multiexp: handle pippenger multiexps with part precalc (moneromooo-monero)
10e5a927 bulletproofs: maintain -z4, -z5, and -y0 to avoid subtractions (moneromooo-monero)
8629a42c bulletproofs: rework flow to use sarang's fast batch inversion code (moneromooo-monero)
fc9f7d9c bulletproofs: merge multiexps as per sarang's new python code (moneromooo-monero)
4061960a multiexp: pack the digits table when STRAUS_C is 4 (moneromooo-monero)
bf8e4b98 bulletproofs: some more minor speedup (moneromooo-monero)
c415df97 performance_tests: sc_check and ge_dsm_precomp (moneromooo-monero)
a281b950 bulletproofs: remove single value prover (moneromooo-monero)
484155d0 bulletproofs: some more speedup (moneromooo-monero)
a621d6c8 bulletproofs: random minor speedups (moneromooo-monero)
a49a1761 bulletproofs: shave off a lot of scalar muls from the g/h construction (moneromooo-monero)
4564a5d1 bulletproofs: speedup PROVE (moneromooo-monero)
15:43 < hahsun> Im on stagenet and I suddenly get this exception: 2018-11-04 14:42:52.416 [RPC0] ERROR wallet.wallet2 src/wallet/wallet2.cpp:2070 !m_blockchain.is_in_bounds(current_index). THROW EXCEPTION: error::out_of_hashchain_bounds_error
16:01 <+moneromooo> OK, possibly because the blckchain is always seeded with the genesis block hash...
16:02 <+moneromooo> So that case should be allowed, assuming it doesn't break the code around it.
16:05 <+moneromooo> OK if stop_height == size || (size==1 && stop_heigt ==0)
16:05 <+moneromooo> Throw if not that.
16:06 < hahsun> k
This happens for every historical tx when syncing, and the
unnecessary parsing is actually showing up on profile.
Since these are kept cached for just one block, this does
not increase memory usage after syncing.
It seems the more prudent thing to do here. It will not catch
attempts to use that value before it is initialized when using
ASAN or valgrind, but in a case where it does, it will have
smaller repercussions.
So it seems appropriate in this particular case.
Coverity 182498
