0d9c0db9 Do not build against epee_readline if it was not built (Howard Chu)
178014c9 split off readline code into epee_readline (moneromooo-monero)
a9e14a19 link against readline only for monerod and wallet-wallet-{rpc,cli} (moneromooo-monero)
437421ce wallet: move some scoped_message_writer calls from the libs (moneromooo-monero)
e89994e9 wallet: rejig to avoid prompting in wallet2 (moneromooo-monero)
ec5135e5 move input_line from command_line to simplewallet (moneromooo-monero)
082db75f move cryptonote command line options to cryptonote_core (moneromooo-monero)
ec48e8d8 core: do not forbid txes without destination (moneromooo-monero)
523084bc core: don't add empty additional pub keys field to extra (moneromooo-monero)
This patch allows to filter out sensitive information for queries that rely on the pool state, when running in restricted mode.
This filtering is only applied to data sent back to RPC queries. Results of inline commands typed locally in the daemon are not affected.
In practice, when running with `--restricted-rpc`:
* get_transaction_pool will list relayed transactions with the fields "last relayed time" and "received time" set to zero.
* get_transaction_pool will not list transaction that have do_not_relay set to true, and will not list key images that are used only for such transactions
* get_transaction_pool_hashes.bin will not list such transaction
* get_transaction_pool_stats will not count such transactions in any of the aggregated values that are computed
The implementation does not make filtering the default, so developers should be mindful of this if they add new RPC functionality.
Fixes#2590.
Transactions in the txpool are marked when another transaction
is seen double spending one or more of its inputs.
This is then exposed wherever appropriate.
Note that being marked with this "double spend seen" flag does
NOT mean this transaction IS a double spend and will never be
mined: it just means that the network has seen at least another
transaction spending at least one of the same inputs, so care
should be taken to wait for a few confirmations before acting
upon that transaction (ie, mostly of use for merchants wanting
to accept unconfirmed transactions).
It is safe in those cases, though might return slightly out of date
information if another thread is busy modifying the blockchain,
but it avoids potentially lengthy delays just to get things like
the current blockchain height.
22b51e06 db_lmdb: include chain height when failing to find an output key (moneromooo-monero)
5db433b3 blockchain: avoid exceptions in output verification (moneromooo-monero)
0aaaca29 tx_pool: set the "invalid input" bit when check_tx_inputs fails (moneromooo-monero)
9236823b simplewallet: print tx rejection reason where it was missing (moneromooo-monero)
3dee3301 core_rpc_server: print tx rejection reason at L0 too (moneromooo-monero)
6137a0b9 blockchain: reject unsorted ins and outs from v7 (moneromooo-monero)
16afab90 core: sort ins and outs key key image and public key, respectively (moneromooo-monero)
0c36b9f9 common: add apply_permutation file and function (moneromooo-monero)
It was always returning true, and could not be foreseen to
usefully return errors in the future. This silences CID 162652
as well as saves some checking code in a few places.
0299cb77 Fix various oversights/bugs in ZMQ RPC server code (Thomas Winget)
77986023 json serialization for rpc-relevant monero types (Thomas Winget)
5c1e08fe Refactor some things into more composable (smaller) functions (Thomas Winget)
9ac2ad07 DRY refactoring (Thomas Winget)
And optimize import startup:
Remember start_height position during initial count_blocks pass
to avoid having to reread entire file again to arrive at start_height
It is unused, as it was apparently a future optimization,
and it leaks some information (though since pools publish
thei blocks they find, that amount seems small).
Structured {de-,}serialization methods for (many new) types
which are used for requests or responses in the RPC.
New types include RPC requests and responses, and structs which compose
types within those.
# Conflicts:
# src/cryptonote_core/blockchain.cpp
This commit refactors some of the rpc-related functions in the
Blockchain class to be more composable. This change was made
in order to make implementing the new zmq rpc easier without
trampling on the old rpc.
New functions:
Blockchain::get_num_mature_outputs
Blockchain::get_random_outputs
Blockchain::get_output_key
Blockchain::get_output_key_mask_unlocked
Blockchain::find_blockchain_supplement (overload)
functions which previously had this functionality inline now call these
functions as necessary.
This might prevent some calls to terminate when the LockedTXN
dtor is called as part of stack unwinding caused by another
exception in the first place.
c867357a cryptonote_protocol: error handling on cleanup_handle_incoming_blocks (moneromooo-monero)
ce901fcb Fix blockchain_import wedge on exception in cleanup_handle_incoming_blocks (moneromooo-monero)
84fa015e core: guard against exceptions in handle_incoming_{block,tx} (moneromooo-monero)
5807529e blockchain: cap memory size of retrieved blocks (moneromooo-monero)
c1b10381 rpc: decrease memory usage a bit in getblocks.bin (moneromooo-monero)
3dd34a49 Cleanup test impact of moving blockchain_db_types() (Howard Chu)
80344740 More DB support cleanup (Howard Chu)
4c7f8ac0 DB cleanup (Howard Chu)
If monerod is started with default sync mode, set it to SAFE after
synchronization completes. Set it back to FAST if synchronization
restarts (e.g. because another peer has a longer blockchain).
If monerod is started with an explicit sync mode, none of this
automation takes effect.
5d4ef719 core: speed up output index unique set calculation (moneromooo-monero)
19d7f568 perf_timer: allow profiling more granular than millisecond (moneromooo-monero)
bda8c598 epee: add nanosecond timer and pause/restart profiling macros (moneromooo-monero)
Add get_fork_version and add_ideal_fork_version to core so
cryptonote_protocol does not have to need the Blockchain
class directly, as it's not in its dependencies, and add
those to the fake core classes in tests too.
We won't even talk to a peer which claims a wrong version
for its top block. This will avoid syncing to known bad
peers in the first place.
Also add IP fails when failing to verify a block.
A sort+uniq step was done for every tx in a 200 block chunk,
causing a lot of repeated scanning as the size of the offset
map got larger with every added tx. We now do the step only
once at the end of the loop.
Doing it this way potentially uses more memory, but testing
shows that it's currently only about 2% more.
If the number of blocks to check was not a multiple of the
number of preparation threads, the last few blocks would
not be included in the threaded long hash calculation.
Those would still get calculated when the block gets added
to the chain, however, so this was only a tiny performance
hit, rather than a security bug.
They used to be sorted by amount, which was fine before rct,
but is now suboptimal, since amounts are not known anymore.
In particular, it would give a recipient knowledge of whether
change was higher or lower than the amount received.
02d66db4 tx_pool: initialize padding in txpool meta structure (moneromooo-monero)
0722aea3 cryptonote_core: initialize checkpoint flag (moneromooo-monero)
91d41090 tx_pool: ensure txes loaded from poolstate.bin have their txid cached (moneromooo-monero)
aaeb164c tx_pool: remove transactions if they're in the blockchain (moneromooo-monero)
558cfc31 core, wallet: faster tx pool scanning (moneromooo-monero)
f065234b core: cache tx and block hashes in the respective classes (moneromooo-monero)
The txid is not saved, and we want to make sure the transactions
have their txid cached while in the pool, since get_transactions
copies the transaction object, so any txid calculation on those
copies would not benefit any later caller, since the original tx
would be left without a cached txid.
Minimum mixin 4 and enforced ringct is moved from v5 to v6.
v5 is now used for an increased minimum block size (from 60000
to 300000) to cater for larger typical/minimum transaction size.
The fee algorithm is also changed to decrease the base per kB
fee, and add a cheap tier for those transactions which we do
not care if they get delayed (or even included in a block).
b553c282 rpc: fix BUILD_TAG mispelling (BUILDTAG) (moneromooo-monero)
02097c87 core: print the "new update found" message in cyan, for visibility (moneromooo-monero)
749ebace download: check available disk space before downloading (moneromooo-monero)
f36c5f1e download: give download threads distinct names (moneromooo-monero)
f6211322 core: make update download cancellable (moneromooo-monero)
63f0e074 download: async API (moneromooo-monero)
9bf017ed http_client: allow cancelling a download (moneromooo-monero)
0d90123c http_client: allow derived class to get headers at start (moneromooo-monero)
If the blocks aren't being linked against a binary (such as
one of the blockchain utilities), the symbol will not be
NULL, but the size will be 0. This avoids a apurious warning
about the data hash.
This is to prevent unbounded memory use. Since I don't think there
is a container that has quick insert, quick lookup, and automatic
FIFO, I use two and swap every N, clearing the oldest one.
You're wondering how this fixes core tests, aren't you...
It prevents the miner (initialized by cryptonote::core) from
breaking trying to access arguments that were not added.
Since the tests don't use the miner directly, it makes more
sense to have cryptonote_core add those, since it also uses
the miner.
They're now used by core to determine the data directory to use
for the txpool directory.
This fixes an assert in the core tests, which don't use the RPC
server, which normally initializes the P2P code.
a427235e core: add a missing newline on a string to be logged (moneromooo-monero)
b6a2230e unit_tests: fix minor blockchain_db regression (moneromooo-monero)
c488eca5 hardfork: tone down some logs (moneromooo-monero)
bed2d9f2 Get rid of directory lock (Howard Chu)
2e913676 Handle map resizes from other processes (Howard Chu)
bf1348b7 Can't cache num_txs or num_outputs either (Howard Chu)
dc53e9ee Add a few read txns to streamline (Howard Chu)
When scanning for outputs used in a set of incoming blocks,
we expect that some of the inputs in their transactions will
not be found in the blockchain, as they could be in previous
blocks in that set. Those outputs will be scanned there at
a later point. In this case, we add a flag to control wehther
an output not being found is expected or not.
Zero change is sent to a random address, which confuses the code
which determines which key to use to encrypt the payment id.
Ignore zero amounts for this purpose, so the payment id gets
encrypted with the real destination's key.
3ae79a59 core: set missing verifivation_failed flag when rejecting a tx (moneromooo-monero)
ea6549e9 core_tests: decrease trace level from trace to debug (moneromooo-monero)
0644eed7 Remove boost/foreach.cpp includes (Miguel Herranz)
36dd3e23 Replace BOOST_REVERSE_FOREACH with ranged for (Miguel Herranz)
629e3101 Replace BOOST_FOREACH with C++11 ranged for (Miguel Herranz)
poolstate.bin and p2pstate.bin are stored in .bitmonero/ if the default
P2P port is being used.
If another port is used both files are stored in
.bitmonero/PORTNUMBER/.
This replaces the epee and data_loggers logging systems with
a single one, and also adds filename:line and explicit severity
levels. Categories may be defined, and logging severity set
by category (or set of categories). epee style 0-4 log level
maps to a sensible severity configuration. Log files now also
rotate when reaching 100 MB.
To select which logs to output, use the MONERO_LOGS environment
variable, with a comma separated list of categories (globs are
supported), with their requested severity level after a colon.
If a log matches more than one such setting, the last one in
the configuration string applies. A few examples:
This one is (mostly) silent, only outputting fatal errors:
MONERO_LOGS=*:FATAL
This one is very verbose:
MONERO_LOGS=*:TRACE
This one is totally silent (logwise):
MONERO_LOGS=""
This one outputs all errors and warnings, except for the
"verify" category, which prints just fatal errors (the verify
category is used for logs about incoming transactions and
blocks, and it is expected that some/many will fail to verify,
hence we don't want the spam):
MONERO_LOGS=*:WARNING,verify:FATAL
Log levels are, in decreasing order of priority:
FATAL, ERROR, WARNING, INFO, DEBUG, TRACE
Subcategories may be added using prefixes and globs. This
example will output net.p2p logs at the TRACE level, but all
other net* logs only at INFO:
MONERO_LOGS=*:ERROR,net*:INFO,net.p2p:TRACE
Logs which are intended for the user (which Monero was using
a lot through epee, but really isn't a nice way to go things)
should use the "global" category. There are a few helper macros
for using this category, eg: MGINFO("this shows up by default")
or MGINFO_RED("this is red"), to try to keep a similar look
and feel for now.
Existing epee log macros still exist, and map to the new log
levels, but since they're used as a "user facing" UI element
as much as a logging system, they often don't map well to log
severities (ie, a log level 0 log may be an error, or may be
something we want the user to see, such as an important info).
In those cases, I tried to use the new macros. In other cases,
I left the existing macros in. When modifying logs, it is
probably best to switch to the new macros with explicit levels.
The --log-level options and set_log commands now also accept
category settings, in addition to the epee style log levels.
3ff54bdd Check for correct thread before ending batch transaction (Howard Chu)
eaf8470b Must wait for previous batch to finish before starting new one (Howard Chu)
c903c554 Don't cache block height, always get from DB (Howard Chu)
eb1fb601 Tweak default db-sync-mode to fast:async:1 (Howard Chu)
0693cff9 Use batch transactions when syncing (Howard Chu)
fsync the DB asynchronously, to allow block download/verification
to proceed while syncing. Sync after every batch. Note that
"fastest" still defaults to fastest:async:1000.
It can now be queried by RPC, so it needs to be set before
it is otherwise needed for consensus, even if no blocks had
to be added (ie, exit and restart quickly).
Continue filling until we reach the block size limit, or the
resulting coinbase decreases.
Also remove old sanity check on block size, which is now not
wanted anymore.
After popping blocks from the old chain, the hard fork object's
notion of the current version was not in line with the new height,
causing the first blocks from the new chain to be rejected due
to a false expection of a newer version.
A bug in cold signing caused a spurious pubkey to be included
in transactions, so we need to ensure we use the correct one
when sending outputs from one of those.
If the block reward to use for the fee calculation can't be
calculated (should not happen in practice), use a high bound,
so we use a fee overestimate that will be accepted by the network.
The vast majority of transactions will have just one tx pubkey,
but a bug with cold wallet signing caused two such keys to be
there, with the second one being the real one.
Added a new command to the P2P protocol definitions to allow querying for support flags.
Implemented handling of new support flags command in net_node. Changed for_each callback template to include support flags. Updated print_connections command to show peer support flags.
Added p2p constant for signaling fluffy block support.
Added get_pool_transaction function to cryptnote_core.
Added new commands to cryptonote protocol for relaying fluffy blocks.
Implemented handling of fluffy block command in cryptonote protocol.
Enabled fluffy block support in node initial configuration.
Implemented get_testnet function in cryptonote_core.
Made it so that fluffy blocks only run on testnet.
Compute derivation only once per tx, instead of once per output. Approx 33% faster while using 75% as much CPU on my machine. Note old functions in cryptonote_core (lookup_acc_outs and is_out_to_acc) are still used by tests.
The fee will vary based on the base reward and the current
block size limit:
fee = (R/R0) * (M0/M) * F0
R: base reward
R0: reference base reward (10 monero)
M: block size limit
M0: minimum block size limit (60000)
F0: 0.002 monero
Starts applying at v4
25% of the outputs are selected from the last 5 days (if possible),
in order to avoid the common case of sending recently received
outputs again. 25% and 5 days are subject to review later, since
it's just a wallet level change.
4038e86 Add performance timers for ringct tx verification (moneromooo-monero)
74dfdb0 perf_timer: new class and macros to make performance logs easier (moneromooo-monero)
10be903 Brackets to prevent premature return (NanoAkron)
fb1785a Brackets to ensure doesn't function prematurely return (NanoAkron)
8ed0d72 Moved logging to target functions rather than caller (NanoAkron)
442bfd1 Added messages at log level 2 to reflect deactivation procedure (NanoAkron)
bba6af9 wallet: cold wallet transaction signing (moneromooo-monero)
9872dcb wallet: fix log confusion between bytes and kilobytes (moneromooo-monero)
d9b0bf9 cryptonote_core: make extra field removal more generic (moneromooo-monero)
98f19d4 serialization: add support for serializing std::pair and std::list (moneromooo-monero)
This change adds the ability to create a new unsigned transaction
from a watch only wallet, and save it to a file. This file can
then be moved to another computer/VM where a cold wallet may load
it, sign it, and save it. That cold wallet does not need to have
a blockchain nor daemon. The signed transaction file can then be
moved back to the watch only wallet, which can load it and send
it to the daemon.
Two new simplewallet commands to use it:
sign_transfer (on the cold wallet)
submit_transfer (on the watch only wallet)
The transfer command used on a watch only wallet now writes an
unsigned transaction set in a file called 'unsigned_monero_tx'
instead of submitting the tx to the daemon as a normal wallet does.
The signed tx file is called 'signed_monero_tx'.
Keep the immediate direct deps at the library that depends on them,
declare deps as PUBLIC so that targets that link against that library
get the library's deps as transitive deps.
Break dep cycle between blockchain_db <-> crytonote_core.
No code refactoring, just hide cycle from cmake so that
it doesn't complain (cycles are allowed only between
static libs, not shared libs).
This is in preparation for supproting BUILD_SHARED_LIBS cmake
built-in option for building internal libs as shared.
Since this queries block heights for blocks that may or may not
exist, queries for non existing blocks would throw an exception,
and that would slow down the loop a lot. 7 seconds to go through
a 30 hash list.
Fix this by adding an optional return block height to block_exists
and using this instead. Actual errors will still throw an
exception.
This also cuts down on log exception spam.
The code used to cap at 5000 blocks per sync. It also treated 0 as 1.
Remove these checks; if specified as 0 do no periodic syncs at all.
Then the user is responsible for syncing in some external process.
When RingCT is enabled, outputs from coinbase transactions
are created as a single output, and stored as RingCT output,
with a fake mask. Their amount is not hidden on the blockchain
itself, but they are then able to be used as fake inputs in
a RingCT ring. Since the output amounts are hidden, their
"dustiness" is not an obstacle anymore to mixing, and this
makes the coinbase transactions a lot smaller, as well as
helping the TXO set to grow more slowly.
Also add a new "Null" type of rct signature, which decreases
the size required when no signatures are to be stored, as
in a coinbase tx.
This allows the key to be not the same for two outputs sent to
the same address (eg, if you pay yourself, and also get change
back). Also remove the key amounts lists and return parameters
since we don't actually generate random ones, so we don't need
to save them as we can recalculate them when needed if we have
the correct keys.
The whole rct data apart from the MLSAGs is now included in
the signed message, to avoid malleability issues.
Instead of passing the data that's not serialized as extra
parameters to the verification API, the transaction is modified
to fill all that information. This means the transaction can
not be const anymore, but it cleaner in other ways.
Since these are needed at the same time as the output pubkeys,
this is a whole lot faster, and takes less space. Only outputs
of 0 amount store the commitment. When reading other outputs,
a fake commitment is regenerated on the fly. This avoids having
to rewrite the database to add space for fake commitments for
existing outputs.
This code relies on two things:
- LMDB must support fixed size records per key, rather than
per database (ie, all records on key 0 are the same size, all
records for non 0 keys are same size, but records from key 0
and non 0 keys do have different sizes).
- the commitment must be directly after the rest of the data
in outkey and output_data_t.
The mixRing (output keys and commitments) and II fields (key images)
can be reconstructed from vin data.
This saves some modest amount of space in the tx.
It may be suboptimal, but it's a pain to have to rebuild everything
when some of this changes.
Also, no clue why there seems to be two different code paths for
serializing a tx...
