Since this queries block heights for blocks that may or may not
exist, queries for non existing blocks would throw an exception,
and that would slow down the loop a lot. 7 seconds to go through
a 30 hash list.
Fix this by adding an optional return block height to block_exists
and using this instead. Actual errors will still throw an
exception.
This also cuts down on log exception spam.
CMake issued a warming about policy CMP0026: access of LOCATION
target property at config time was disallowed. Offending code
was the code that merged static libraries to generate
libwallet_merged.a.
This patch does that same merge task in a much simpler way. And,
since it doesn't violate the policy, the warning went away.
The code used to cap at 5000 blocks per sync. It also treated 0 as 1.
Remove these checks; if specified as 0 do no periodic syncs at all.
Then the user is responsible for syncing in some external process.
When RingCT is enabled, outputs from coinbase transactions
are created as a single output, and stored as RingCT output,
with a fake mask. Their amount is not hidden on the blockchain
itself, but they are then able to be used as fake inputs in
a RingCT ring. Since the output amounts are hidden, their
"dustiness" is not an obstacle anymore to mixing, and this
makes the coinbase transactions a lot smaller, as well as
helping the TXO set to grow more slowly.
Also add a new "Null" type of rct signature, which decreases
the size required when no signatures are to be stored, as
in a coinbase tx.
This allows the key to be not the same for two outputs sent to
the same address (eg, if you pay yourself, and also get change
back). Also remove the key amounts lists and return parameters
since we don't actually generate random ones, so we don't need
to save them as we can recalculate them when needed if we have
the correct keys.
The whole rct data apart from the MLSAGs is now included in
the signed message, to avoid malleability issues.
Instead of passing the data that's not serialized as extra
parameters to the verification API, the transaction is modified
to fill all that information. This means the transaction can
not be const anymore, but it cleaner in other ways.
This plugs a privacy leak, where the wallet tells the daemon
which transactions contain outputs for the wallet by asking
for additional information for that particular transaction.
As a nice bonus, this actually makes refresh slightly faster.
With RCT, we allow 0 size outputs, to try and encourage txes
with two inputs and two outputs. Consolidation would then
have two non zero inputs, one zero output, and one larger
output.
Before the normal selection, we attempt to find either one or two
suitable outputs to use as inputs to the rct tx. The intent is that
most rct txes will have one or two inputs, and we want all to look
the same if possible.
When two outputs are needed, we try to find a pair which are not
related (ie, by being from the same or similar block height).
The "transfer" simplewallet command is renamed to "transfer_original".
"transfer_new" is renamed "transfer", "transfer_rct" is removed,
and the new "transfer" now selects rct or non rct transactions
based on the current block height.
Since these are needed at the same time as the output pubkeys,
this is a whole lot faster, and takes less space. Only outputs
of 0 amount store the commitment. When reading other outputs,
a fake commitment is regenerated on the fly. This avoids having
to rewrite the database to add space for fake commitments for
existing outputs.
This code relies on two things:
- LMDB must support fixed size records per key, rather than
per database (ie, all records on key 0 are the same size, all
records for non 0 keys are same size, but records from key 0
and non 0 keys do have different sizes).
- the commitment must be directly after the rest of the data
in outkey and output_data_t.
The mixRing (output keys and commitments) and II fields (key images)
can be reconstructed from vin data.
This saves some modest amount of space in the tx.
If the blockchain gets reorganized, all outputs spent in the part
of the blockchain that's blown away need to be reset to unspent
(they may end up spent again on the blocks that replace the blocks
that are removed, however).
It may be suboptimal, but it's a pain to have to rebuild everything
when some of this changes.
Also, no clue why there seems to be two different code paths for
serializing a tx...
A new version of genRct takes the mixRing as parameter, instead
of the inPk. inPk are part of the mixRing, and it is cleaner to
pass the mixRing data than to fetch it from the RingCT code.
A new version of decodeRct also returns the mask.
Also, failure to decode throws, so errors are properly detected.
This lets my gcc picks those instead of the generic template
where appropriate (and then fail since std::vector<something>
does not have a serialize method.
99dd572 libwallet_api: tests: checking for result while opening wallet (Ilya Kitaev)
bcf7b67 libwallet_api: Wallet::amountFromString fixed (Ilya Kitaev)
32bc7b4 libwallet_api: helper method to return maximumAllowedAmount (Ilya Kitaev)
cbe534d libwallet_api: tests: removed logged passwords (Ilya Kitaev)
b1a5a93 libwallet_api: do not store wallet on close if status is not ok (Ilya Kitaev)
This plugs a privacy leak from the wallet to the daemon,
as the daemon could previously see what input is included
as a transaction input, which the daemon hadn't previously
supplied. Now, the wallet requests a particular set of
outputs, including the real one.
This can result in transactions that can't be accepted if
the wallet happens to select too many outputs with non standard
unlock times. The daemon could know this and select another
output, but the wallet is blind to it. It's currently very
unlikely since I don't think anything uses non default
unlock times. The wallet requests more outputs than necessary
so it can use spares if any of the returns outputs are still
locked. If there are not enough spares to reach the desired
mixin, the transaction will fail.
f0c0a3fFix#864 Squashed commit of the following: commit 9af9e4223b58bbb65a3519af2c2bfc273cbd23d6 fixed some formatting commit c7920e1cf88ff46eb9294101344d9a567f22e2da Merge: 97eb28b 1da1c68 fix#864 fix using boolean commit 97eb28ba5dd49ddde8c8785f39b24d955e5de31c Fix#864 boolean value used to verify on new wallet commit 1da1c68bd3a9a373c70482b6e6e95251096149f1 fix#864 changed to boolean to prompt for verify commit 5bee96652434762d2c91ce31a1b1c9f169446ddc fix 864; made variable names easier for understanding branching. commit 45715960d30293f781b2ff9e5e647c2ec893f4a3 fix#864; allow password to be entered twice for new wallets for verification. fix#864 password entry verification; ammended boolean fix#864 ; default constructor for password_container should set verify=true (guzzi_jones)
Simplewallet improperly skipped the restore from height code if
restoring a deterministic wallet AND not specifying a wallet file in the
command line. The other generate options require a wallet file as an
argument, which prevents "ask_wallet_create_if_needed()" from being
called, which in turn causes "m_generate_new" to remain unset.
Specifying a wallet file at launch with --restore-deterministic emulated
this behavior.
This constrains the number of instances of any amount
to the unlocked ones (as defined by the default unlock time
setting: outputs with non default unlock time are not
considered, so may be counted as unlocked even if they are
not actually unlocked).
Squashed commit of the following:
commit 9af9e4223b58bbb65a3519af2c2bfc273cbd23d6
fixed some formatting
commit c7920e1cf88ff46eb9294101344d9a567f22e2da
Merge: 97eb28b 1da1c68
fix#864 fix using boolean
commit 97eb28ba5dd49ddde8c8785f39b24d955e5de31c
Fix#864 boolean value used to verify on new wallet
commit 1da1c68bd3a9a373c70482b6e6e95251096149f1
fix#864 changed to boolean to prompt for verify
commit 5bee96652434762d2c91ce31a1b1c9f169446ddc
fix 864; made variable names easier for understanding branching.
commit 45715960d30293f781b2ff9e5e647c2ec893f4a3
fix#864; allow password to be entered twice for new wallets for verification.
fix#864 password entry verification; ammended boolean
fix#864 ; default constructor for password_container should set verify=true
The previous logic that used a COMMON_*_FLAGS intermediate variable
and then re-assigned CMAKE_*_FLAGS before including each subdirectory
was confusing and ugly. This PR is the right way to do it.
This commit is purely refactoring: built binaries unchanged.
By default the flag is enabled whenever libunwind is found on the
system, with the exception of static build on OSX (for which we can't
install the throw hook #932 due to lack of support for --wrap in OSX
ld64 linker).
They are used to export a signed set of key images from a wallet
with a private spend key, so an auditor with the matching view key
may see which of those are spent, and which are not.
It is not clear why libunbound was added to this in the first place,
since it wasn't here before and #915 doesn't seem to introduce any
new dependency on it.
Tested build with STATIC=OFF (with and without libunbound-dev libunbound8
installed) and STATIC=ON, on Ubuntu Trusty, Debian Jessie, and Arch
Linux. For static builds, beware of #926 and #907.
If this hack was introduced to make it build on some other system
(Windows? OS X?), then it will have to be dealt with, but not this way.
f07f120 cmake: don't try to link with atomic on Apple (redfish)
19349d7 cmake: ARM: clang: make warning non-fatal: inline asm (redfish)
f3e09f3 cmake: link with -latomic for clang (redfish)
f4b35ae cmake: include -ldl via cmake built-in var (redfish)
fa85cd8 common: stack trace: make clang happy with func ptrs (redfish)
4dce26b cmake: do not pass -stdlib=c++ to clang >=3.7 (redfish)
78cc10f daemon: fix ban seconds being misinterpreted as absolute (moneromooo-monero)
34ecfdb rpc: fix get_bans and set_bans RPC names, they were missing a _ (moneromooo-monero)
Signing is done using the spend key, since the view key may
be shared. This could be extended later, to let the user choose
which key (even a per tx key).
simplewallet's sign/verify API uses a file. The RPC uses a
string (simplewallet can't easily do strings since commands
receive a tokenized set of arguments).
Tested that it builds with:
gcc 6.1.1, STATIC=OFF,i686
gcc 6.1.1, STATIC=OFF,armv7h
clang 3.8, STATIC=OFF,i686
clang 3.8, STATIC=OFF,armv7h
gcc 6.1.1, STATIC=ON,i686
clang 3.8, STATIC=ON,i686
Also tested that stack trace is generated fine on exception on:
i686, gcc 6.1.1, STATIC=OFF
(didn't bother testing all the other platforms/configs)
This should fix the build problem on OSX (#871, #901), but
I don't have OSX, so I could only test Clang on Linux.
Keep the working directory (and umask) inherited from
the parent. Otherwise, it's impossible to control
the working directory of the daemon (from systemd, for
example).
Furthermoer, bitmonerod attempts to create logging directories and files
*in current working directory*. This fails due to permission denied and
generates a (caught, nonfatal) exception. Below is the strace with this
patch applied (so, no `chdir("/")`), showing successful opens at `log/`
relative path. Without this patch they fail (sorry, didn't save the
trace).
```
28911 getcwd("/.../bitmonero", 128) = 25
28911 stat64("/var/lib/bitmonero/.bitmonero", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
28911 stat64("/etc/bitmonerod.conf", {st_mode=S_IFREG|0644, st_size=244, ...}) = 0
28911 open("/etc/bitmonerod.conf", O_RDONLY|O_LARGEFILE) = 3
28911 open("/var/log/bitmonero/bitmonero.log", O_WRONLY|O_CREAT|O_APPEND|O_LARGEFILE, 0666) = 3
28911 stat64("log", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
28911 stat64("log/dbg", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
28911 open("log/dbg/main.log", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 4
```
The reasoning of chdir("/") in order to prevent the daemon from holding
a filesystem in busy state is not compelling at all: the choice of
working directory for the daemon is the user's business not the
daemon's.
When an exception happens while reading the config file, we need
to print the error, as the logging system isn't initialized yet,
so the generic catch will not print anything.
Fee can now be multiplied by 2 or 3, if users want to give
priority to their transactions. There are only three levels
to avoid too much fingerprinting. Default is 1 (minimum fee).
The default multiplier can be set by "set fee-multiplier X".
The former was a faulty "fix" for gmtime_r not existing on Windows. The latter is needed only for dynamic builds, and is not included with msys2, which ends up fine because Windows is only built static at this time.
5dc09f2 wallet_rpc_server: fix some string values being returned between <> (moneromooo-monero)
f8213c0 Require 64/16 characters for payment ids (moneromooo-monero)
c5f00bc cn_deserialize: add minergate data (moneromooo-monero)
2470b20 cn_deserialize: print extra nonce as hex data (moneromooo-monero)
77d8fc3 tx_extra: parse new chunk added by minergate (blashyrkh)
The destructors get a noexcept(true) spec by default, but these
destructors in fact throw exceptions. An alternative fix might be to not
throw (most if not all of these throws are non-essential
error-reporting/logging).
1c0bffb Restrict also 'get_connections' and 'getbans' APIs. (osensei)
9f8bc49 Don't allow 'flush_txpool' and 'setbans' JSON_RPC methods when running in restricted mode. (osensei)
The tests for rejection of unmixable outputs in v2 are commented out,
as there are no unmixable outputs created anymore. This should be
restored at some point.
It sets the max number of threads to use for a parallel job.
This is different that the number of total threads, since monero
binaries typically start a lot of them.
It allows a simple get_transfers (with default 0 min_height and
max_height) to return all transactions, instead of the unexpected
set of txes in block 0, which is probably none at all.
We want to lock operations which access the blockchain in
wallet2. We also want the background refresh to happen again
when we cancel a foreground refresh. Wrap the locking setup
in a macro so it doesn't get copy/pasted/mangled, and use
a scope exit trick to ensure it's always properly restored.
This sends all outputs in a wallet to a given address, alleviating
the difficulty people have had trying to send all monero but
being left with some small amount left.
modified: src/wallet/wallet2.cpp
modified: src/wallet/wallet2.h
Update to fix unconfirmed balance and give a slightly more verbose and informative confirmation message for transfers
When m_refresh_from_block_height has been set, only hashes will be
retrieved up to that height, instead of full blocks. The same will
be done for "refresh <height>" when the specified height is beyond
the current local blockchain.
When reaching the tail emission phase, the amount of coins will
eventually go over MONEY_SUPPLY, overflowing 64 bits. There was
a check added to blockchain_storage, but this was not ported to
the blockchain DB version.
Reported by smooth.
d662ab5 rpc: print human readable time since received when printing pool (moneromooo-monero)
5c9dd23 rpc: add a do_not_relay boolean to tx submission (moneromooo-monero)
^C when in RPC mode would not save the wallet while it was still
refreshing after starting up.
Also, save the wallet out of the signal handler. We don't want
to call complex stuff in a signal handler.
Also bumped DB VERSION to 1
Another significant speedup and space savings:
Get rid of global_output_indices, remove indirection from output to keys
This is the change warptangent described on irc but never got to finish.
This speeds up wallet refresh by directly retrieving a tx's amount output indices.
It removes the indirection and walking the amount output duplicate list
for every amount in each requested tx.
"tx_outputs" is used by:
Amount output indices are needed for wallet refresh.
Global output indices are needed for removing a tx.
Both amount output indices and global output indices are now stored in
an array of 64-bit unsigned ints:
tx_outputs[<tx_hash>] -> [ <a1_oi, a1_gi, a2_oi, a2_gi, ...> ]
Previously it was:
tx_outputs[<tx_hash>] -> duplicate list of <a1_gi, a2_gi, a3_gi, ...>
The amount output list had to be walked for every amount in order to
find each amount's output index, by comparing the amount's global output
index with each one in the duplicate list until a match was found.
See also d045dfa7ce
b4eada9 wallet: make load_keys check types when loading JSON (moneromooo-monero)
3e55725 wallet: make the JSON reading type safe (moneromooo-monero)
f8d05f3 common: new json_util.h (moneromooo-monero)
This can generate non decomposed outputs for very large block
rewards (or not so large ones if a miner decides to not quantize
the block rewards). Out of an abundance of caution, we refuse
to generate those. They are still accepted by the consensus code,
however.
d5d46e6 tests: obligatory hardfork unit build fix after interface change (moneromooo-monero)
25672d3 wallet: pass std::function by const ref, not value (moneromooo-monero)
0be6e08 wallet: do not leak owned amounts to the daemon unless --trusted-daemon (moneromooo-monero)
12146da wallet: change sweep_dust to sweep_unmixable (moneromooo-monero)
600a3cf New RPC and daemon command to get output histogram (moneromooo-monero)
f9a2fd2 wallet: handle rare case where fee adjustment can bump to the next kB (moneromooo-monero)
f26651a wallet: factor fee calculation (moneromooo-monero)
This allows appropriate action to be taken, like displaying
the reason to the user.
Do just that in simplewallet, which should help a lot in
determining why users fail to send.
Also make it so a tx which is accepted but not relayed is
seen as a success rather than a failure.
With the change in mixin rules for v2, the "annoying" outputs are
slightly changed. There is high correlation between dust and
unmixable, but no equivalence.
This is a list of existing output amounts along with the number
of outputs of that amount in the blockchain.
The daemon command takes:
- no parameters: all outputs with at least 3 instances
- one parameter: all outputs with at least that many instances
- two parameters: all outputs within that many instances
The default starts at 3 to avoid massive spamming of all dust
outputs in the blockchain, and is the current minimum mixin
requirement.
An optional vector of amounts may be passed, to request
histogram only for those outputs.
This was meant to go in v2, but the miner tx slipped through
the cracks as it doesn't go through the main tx verification
since it doesn't get added to the pool.
It takes a filename containing JSON data to generate a wallet.
The following fields are valid:
version: integer, should be 1
filename: string, path/filename for the newly created wallet
scan_from_height: 64 bit unsigned integer, optional
password: string, optional
viewkey: string, hex representation
spendkey: string, hex representation
seed: string, optional, list of words separated by spaces
Either seed or private keys should be given. If using private
keys, the spend key may be omitted (the wallet will not be
able to spend, but will see incoming transactions).
If scan_from_height is given, blocks below this height will not
be checked for transactions as an optimization.
tx_pool.h doxygen documentation completed.
Many notes made on areas for improvement, be that functionality or
code clarity.
Commented code and unused code removed.
The functions in src/cryptonote_core/checkpoints_create.{h,cpp} should
be member functions of the checkpoints class, if nothing else for the
sake of keeping their documentation together.
This commit covers moving those functions to be member functions of the
checkpoints class as well as documenting those functions.
All functions in src/cryptonote_core/checkpoints.h are now documented in
doxygen style.
checkpoints.cpp has been reviewed, one function has been marked for
discussion on correctness.
All functions are now documented in doxygen format. Comments have been
updated to reflect the current state of the code. Many areas for
improvement in clarity and design have been noted, as well as cruft to
be removed. These changes are not reflected in this commit both to
allow time for comment and to keep commits organized by purpose.
Example of current return for `print_block 912345`:
timestamp: 1452793716
previous hash:
b61c58b2e0be53fad5ef9d9731a55e8a81d972b8d90ed07c04fd37ca6403ff78
nonce: 1646
is orphan: 0
height: 912345
depth: 85434
hash:
e22cf75f39ae720e8b71b3d120a5ac03f0db50bba6379e2850975b4859190bc6difficul
ty: 815625611
reward: 7388968946286
{
"major_version": 1,
"minor_version": 2,
…
Without `std::endl`, the difficulty gets smashed on the end of the hash.
It would try to join the auto refresh thread, which would
only happen after it was done, which would take a long time
when doing so on a newly created wallet.
8bc1bd6 wallet: use minimum mixin when RPC asks for too low mixin (moneromooo-monero)
31d2e0f wallet_rpc_server: make use_fork_rules public (moneromooo-monero)
After the fork, normal transfer functions called via RPC
use the minimum mixin 2 if 0 or 1 is requested. While the
incoming transaction may be valid (eg, it has an unmixable
and at most a mixable input), it is a simple way to make
sure RPC users can't get a seemingly random accept/reject
behavior if they don't update their requested mixin.
bdec7cb BlockchainLMDB: Use DB error helper consistently (warptangent)
c5932eb BlockchainLMDB: Add DB error to exception (warptangent)
a49c355 Blockchain: Omit verbose time stats messages by default (warptangent)
TEST:
blockchain_export -h
This should show "berkeley" as an available option to --database.
With an existing BerkeleyDB database, run:
blockchain_export --database berkeley
This is already the default for the daemon, but by checking a command
line argument and calling a Blockchain member function setter.
Initialize the variable to false so it's not dependent on an external
command-line argument check. This allows utilities like
blockchain_import to have a reasonable default without code changes.
c2a1fee simplewallet: prompt for private keys when generating wallets (moneromooo-monero)
4513b4c simplewallet: add a new --restore-from-keys option (moneromooo-monero)
11d555c Fix crash in std::map for connections_map (Howard Chu)
014f886 std::condvar is broken on Win32 with gcc/g++ 4.8 too (Howard Chu)
7c86c59 Use boost::thread instead of std::thread (Howard Chu)
57e75fa BlockchainBDB: Check if hard fork subdbs need reset (warptangent)
47f6cf8 BlockchainBDB: Support blockchain_import --drop-hard-fork command (warptangent)
27d4e50 core: check whether an update is needed straight away (moneromooo-monero)
8892173 core: print "update needed" hard fork notifications in red (moneromooo-monero)
Pass the CMake bit width setting to compile flags for blockchain_import
and blockchain_converter.
For LMDB on 32-bit, hyc has found that batch size of 100 appears to be a
good default.
3b13a74 Shutup about VERSION 0 (Howard Chu)
1537477 Use cursor in get_output_key (Howard Chu)
f2faf8c Use MDB_APPEND mode where possible (Howard Chu)
090b548 Use cursors in write txns (Howard Chu)
ed08d21 Keep a running blocksize count (Howard Chu)
0fc9334 Win32 import batchsize tweaks (Howard Chu)
27f76e2 blockchain_import: Build string for db type list (warptangent)
1aa8a9d blockchain_import: Add mode argument representing multiple DB flags (warptangent)
cffc411 blockchain_import: Support BerkeleyDB (warptangent)
19c1aaa blockchain_import: Add database type argument (warptangent)
0fedce0 blockchain_import: Make LMDB-specific names general (warptangent)
When keys are contiguous and monotonically increasing, this gets
denser page utilization (doesn't leave padding in page splits).
Can't be used for keys that are inserted in random order (e.g. hashes)
In total this only saves around 1.5% of space compared to original
DB code. The previous patch accounted for 0.8% savings on its own;
the blocks tables just aren't that big.
These modes match those optionally provided as part of the daemon's
--db-type argument.
Argument after the # is interpreted as a composite mode if there's only
one (no comma separated arguments).
Sample usage:
blockchain_import --database lmdb#fastest
blockchain_import --database berkeley#fastest
Multiple specific DB flags are still supported, e.g.
blockchain_import --database lmdb#nosync,nordahead
blockchain_import --database berkeley#txn_nosync
