Lack of it results in `m_refresh_from_block_height` being < 0 (18446744...) on low heights, which blocks `process_new_blockchain_entry` and never process coins on heights less than blocks_per_month.
Follow-up to #2258
a15e8583 wallet2: guard against daemon sending txes in the wrong order (moneromooo-monero)
8fe5f609 rpc: order transactions in the order they were requested (moneromooo-monero)
d58700e0 WalletAPI: only allow trusted daemon when importing key images (Jaquee)
8a9bbd26 WalletAPI: copy wallet data when creating a view only wallet (Jaquee)
d27fe32e wallet2: export/import wallet data functions (Jaquee)
225a25f3 import_key_images - allow importing without being connected to daemon (Jaquee)
b7d6ec83 simplewallet: add (out of sync) or (no daemon) markers in the prompt (moneromooo-monero)
fa23a500 wallet2: add a is_synced function (moneromooo-monero)
f1307bbd node_rpc_proxy: add a proxy for target height (moneromooo-monero)
When scanning the txpool without having first updated the
blockchain, the tx would be seen as neither in the txpool
nor the chain, and removed, so it'd only reappear once the
chain is refreshed, and the tx seen in a block.
If the refresh height is in the future, the current code will
loop till the actual height reaches this. Fix it by bailing out
if we receive only three hashes, which is what we set in the
call parameters.
The previous patch was based on a wrong premise (that the
daemon height was 0 because the daemon calling code wasn't
yet initialized). In fact, current height approximation
was not setup for testnet. Fix this.
e2529347 Correct spelling of 'get_upper_transaction_size_limit' (Nano Akron)
3029d0ef Remove the 1.25x multiplier in max transaction size in just the wallet (Nano Akron)
It sweeps all outputs below the given threshold
This is available via the existing sweep_all RPC, by setting
amount_threshold the desired amount (in atomic units)
a6d5bb75 wallet2: refer to triangular distribution for recent zone in comment (moneromooo-monero)
ac1aba90 wallet2: bias fake outs more towards recent outputs (moneromooo-monero)
It was wrongly refering to equiprobable distribution, which I think
I'd originally done, but forgot to update the comment after changing
to triangular
Reported by smooth on IRC
Two recent papers quantified the real usage bias for the
real output in a ring being the true one, and shows that
the current biasing is much too weak.
While we wait for a better solution, we increase the ratio
of recent-to-total fake outputs, as well as decrease the
time window for recent outputs, so that half the fake outs
are selected within the last 1.8 day. Value plucked from
figure 10, page 11 of An Empirical Analysis of Linkability
in the Monero Blockchain, 2017, Miller et al.
This is also arbitrary, of course, but serves as a stopgap
till a better selection algorithm is chosen.
If using a large input and many destinations, the code would
generate as many outputs as it could using that input, even if
it would bring the resulting tx above the max tx size.
With the change from the original transfer method to the new
algorithm, payments to the same destination were merged. It
seemed like a good idea, optimizing space. However, it is a
useful tool for people who want to split large outputs into
several smaller ones (ie, service providers making frequent
payments, and who do not like a large chunk of their balance
being locked for 10 blocks after each payment).
Default to off, which is a change from the previous behavior.
When a single input is enough to satisfy a transfer, the code would
previously try to add a second input, to match the "canonical" makeup
of a transaction with two inputs and two outputs. This would cause
wallets to slowly merge outputs till all the monero ends up in a
single output, which causes trouble when making two transactions
one after the other, since change is locked for 10 blocks, and an
increasing portion of the remaining balance would end up locked on
each transaction.
There are two new settings (min-output-count and min-output-value)
which can control when to stop adding such unneeded second outputs.
The idea is that small "dust" outputs will still get added, but
larger ones will not.
Enable with, eg:
set min-output-count 10
set min-output-value 30
to avoid using an unneeded second output of 30 monero or more, if
there would be less than 10 such outputs left.
This does not invalidate any other reason why such outputs would
be used (ie, when they're really needed to satisfy a transfer, or
when randomly picked in the normal course of selection). This may
be improved in the future.
Asking for a full histogram from a remote node (since it's
untrusted) is pretty slow, and spams the remote node, so
we replace it by only adding a second input if we have rct
ones, which are for all intents and purposes always mixable.
Minimum mixin 4 and enforced ringct is moved from v5 to v6.
v5 is now used for an increased minimum block size (from 60000
to 300000) to cater for larger typical/minimum transaction size.
The fee algorithm is also changed to decrease the base per kB
fee, and add a cheap tier for those transactions which we do
not care if they get delayed (or even included in a block).
This reverts commit d47dac9a88.
Callers actually expect the key to be payment id, so this
needs a lot more changes (like storing payment ids in the
structure, and possibly also to other existing structures
which do the same thing).
A relatedness check was meant to be done in the case of adding
an extra output if just one was enough. This was mistakenly
added to the "preferred output" case.
c02e1cb9 Updates to epee HTTP client code - http_simple_client now uses std::chrono for timeouts - http_simple_client accepts timeouts per connect / invoke call - shortened names of epee http invoke functions - invoke command functions only take relative path, connection is not automatically performed (Lee Clagett)
If a rct transaction can be made with just one input, a second
output will be added. This output will be the smallest amount
output available. However, if this output is a non rct output
with less available fake outs than requested, the transaction
will be rejected. We now check the histogram to only consider
outputs with enough available fake outs in the first place.
These warnings were emitted by clang++, and they are real bugs.
src/rpc/core_rpc_server.cpp:208:58: warning: adding 'uint64_t'
(aka 'unsigned long') to a string does not append to the string
[-Wstring-plus-int]
res.status = "Error retrieving block at height " + height;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~
The obvious intent is achieved by using std::to_string().
- http_simple_client now uses std::chrono for timeouts
- http_simple_client accepts timeouts per connect / invoke call
- shortened names of epee http invoke functions
- invoke command functions only take relative path, connection
is not automatically performed
This would have tried to send a second output to make the tx
look like the 2/2 ideal, but it would not fail to find one
because picking an output from preferred_inputs priority list
did not remove it from the unused tranfer/dust outputs, so
it would try to send the same output twice.
While there, I also added a check to avoid sending a second
input if it's related to the first. Better 1/2 than linking
inputs, I think.
This avoids indirectly leaking the real output to the daemon,
and is faster.
This will still happen for more complex cases, especially
when cancelling a tx and "re-rolling" it.
This replaces the epee and data_loggers logging systems with
a single one, and also adds filename:line and explicit severity
levels. Categories may be defined, and logging severity set
by category (or set of categories). epee style 0-4 log level
maps to a sensible severity configuration. Log files now also
rotate when reaching 100 MB.
To select which logs to output, use the MONERO_LOGS environment
variable, with a comma separated list of categories (globs are
supported), with their requested severity level after a colon.
If a log matches more than one such setting, the last one in
the configuration string applies. A few examples:
This one is (mostly) silent, only outputting fatal errors:
MONERO_LOGS=*:FATAL
This one is very verbose:
MONERO_LOGS=*:TRACE
This one is totally silent (logwise):
MONERO_LOGS=""
This one outputs all errors and warnings, except for the
"verify" category, which prints just fatal errors (the verify
category is used for logs about incoming transactions and
blocks, and it is expected that some/many will fail to verify,
hence we don't want the spam):
MONERO_LOGS=*:WARNING,verify:FATAL
Log levels are, in decreasing order of priority:
FATAL, ERROR, WARNING, INFO, DEBUG, TRACE
Subcategories may be added using prefixes and globs. This
example will output net.p2p logs at the TRACE level, but all
other net* logs only at INFO:
MONERO_LOGS=*:ERROR,net*:INFO,net.p2p:TRACE
Logs which are intended for the user (which Monero was using
a lot through epee, but really isn't a nice way to go things)
should use the "global" category. There are a few helper macros
for using this category, eg: MGINFO("this shows up by default")
or MGINFO_RED("this is red"), to try to keep a similar look
and feel for now.
Existing epee log macros still exist, and map to the new log
levels, but since they're used as a "user facing" UI element
as much as a logging system, they often don't map well to log
severities (ie, a log level 0 log may be an error, or may be
something we want the user to see, such as an important info).
In those cases, I tried to use the new macros. In other cases,
I left the existing macros in. When modifying logs, it is
probably best to switch to the new macros with explicit levels.
The --log-level options and set_log commands now also accept
category settings, in addition to the epee style log levels.
If we'd make a rct tx with just one input, we try to add
a second one to match the 2/2 ideal. This means more txes
use that template (and are thus using a larger anonymity
set), and it coalesces outputs "for free". We use the
smallest amount outputs in priority for this, so we can
"clean" the wallet at the same time.
A bug in cold signing caused a spurious pubkey to be included
in transactions, so we need to ensure we use the correct one
when sending outputs from one of those.
If a rct transaction would cause no change to be generated, a zero
change output is added, and sent to a randomly generated address.
This ensures that no transaction will be sent with just one output,
which could cause the receiver to be able to determine which of the
inputs in the sent rings is the real one.
This is very rare, since it requires the sum of outputs to be equal
to the sum of outputs plus the fee, which is now a function of the
last few blocks.
5783dd8c tests: add unit tests for uri parsing (moneromooo-monero)
82ba2108 wallet: add API and RPC to create/parse monero: URIs (moneromooo-monero)
d9001b43 epee: add functions to convert from URL format (ie, %XX values) (moneromooo-monero)
Daemon RPC version is now composed of a major and minor number,
so that incompatible changes bump the major version, while
compatible changes can still bump the minor version without
causing clients to unnecessarily complain.
6d76072 simplewallet: remove double confirmation when submitting signed tx (moneromooo-monero)
92dea04 wallet2: fix wrong change being recorded for cold signed txes (moneromooo-monero)
1d9e223 rpc: do not include output indices for pool txes (moneromooo-monero)
e227d6e rpc: bump version after RPC changes (moneromooo-monero)
2c0173c Add a get_outs (fully text based) version of get_outs.bin (moneromooo-monero)
e05907b rpc: add output indices to gettransactions (moneromooo-monero)
When passing around unsigned and signed transactions, outputs
and key images are passed along (outputs are passed along unsigned
transactions from the hot wallet to the cold wallet, key images
are passed along with signed transations from the cold wallet
to the hot wallet), to allow more user friendly syncing between
hot and cold wallets.
The vast majority of transactions will have just one tx pubkey,
but a bug with cold wallet signing caused two such keys to be
there, with the second one being the real one.
View wallets do not have the spend secret key, and are thus
unable to derive key images for incoming outputs. Moreover,
a previous patch set key images to zero as a means to mark
an output as having an unknown key image, so they could be
filled in when importing key images at a later time. That
later patch caused spurious collisions. We now use public
keys to detect duplicate outputs. Public keys obtained from
the blockchain are checked to be identical to the ones
derived locally, so can't be spoofed.
m_amount_out was sometimes getting initialized with the sum of
an transaction's outputs, and sometimes with the sum of outputs
that were not change. This caused confusion and bugs. We now
always set it to the sum of outputs. This reverts an earlier
fix for bad amounts as this used the other semantics. The wallet
data should be converted automatically in a percentage of cases
that I'm hesitant to estimate. In any case, restoring from seed
or keys or rebuilding the cache will get it right.
Compute derivation only once per tx, instead of once per output. Approx 33% faster while using 75% as much CPU on my machine. Note old functions in cryptonote_core (lookup_acc_outs and is_out_to_acc) are still used by tests.
The intended use is to export outputs from a hot wallet, which
can scan incoming transfers from the network, and import them
in the cold wallet, which can't. The cold wallet can then compute
key images for those outputs, which can then be exported with
export_key_images, etc.
Re-creating the transaction on the cold wallet was not splitting
the change, causing the transaction to be rejected by the network.
This worked on testnet since amounts do not have to be split.
Also add selected_transfers, which can now be saved since they're
size_t rather than iterators. This allows the view wallet to
properly set the sent outputs as spent and update balance.
Bump transfer file version numbers to match.
0950be9 wallet: speed up output selection, and fix bug with relatedness calculation (moneromooo-monero)
0eba133 wallet: fix mixup between mixin 2 and 4 before/after v5 (moneromooo-monero)
25% of the outputs are selected from the last 5 days (if possible),
in order to avoid the common case of sending recently received
outputs again. 25% and 5 days are subject to review later, since
it's just a wallet level change.
This was still using the old transaction creation algorithm,
coupled with a deterministic output selection scheme, which
made it ill suited to the job, since it'd loop indefinitely
in case the fee increased between the test tx and adding the
fee.
bba6af9 wallet: cold wallet transaction signing (moneromooo-monero)
9872dcb wallet: fix log confusion between bytes and kilobytes (moneromooo-monero)
d9b0bf9 cryptonote_core: make extra field removal more generic (moneromooo-monero)
98f19d4 serialization: add support for serializing std::pair and std::list (moneromooo-monero)
This change adds the ability to create a new unsigned transaction
from a watch only wallet, and save it to a file. This file can
then be moved to another computer/VM where a cold wallet may load
it, sign it, and save it. That cold wallet does not need to have
a blockchain nor daemon. The signed transaction file can then be
moved back to the watch only wallet, which can load it and send
it to the daemon.
Two new simplewallet commands to use it:
sign_transfer (on the cold wallet)
submit_transfer (on the watch only wallet)
The transfer command used on a watch only wallet now writes an
unsigned transaction set in a file called 'unsigned_monero_tx'
instead of submitting the tx to the daemon as a normal wallet does.
The signed tx file is called 'signed_monero_tx'.
This fixes misreporting of amount/fee in rct txes, as the rct
tx construction code was lumping all dests (whether change or
not) in the same dests vector, while the pre-rct code was
keeping it separate.
We keep 1, 2, 3 multipliers till the fee decrase from 0.01/kB
to 0.002/kB, where we start using 1, 20, 166 multipliers.
This ensures the higher multiplier will compensate for the
block reward penalty when pushing past 100% of the past median.
The fee-multiplier wallet setting is now rename to priority,
since it keeps its [0..3] range, but maps to different multiplier
values.
When RingCT is enabled, outputs from coinbase transactions
are created as a single output, and stored as RingCT output,
with a fake mask. Their amount is not hidden on the blockchain
itself, but they are then able to be used as fake inputs in
a RingCT ring. Since the output amounts are hidden, their
"dustiness" is not an obstacle anymore to mixing, and this
makes the coinbase transactions a lot smaller, as well as
helping the TXO set to grow more slowly.
Also add a new "Null" type of rct signature, which decreases
the size required when no signatures are to be stored, as
in a coinbase tx.
This allows the key to be not the same for two outputs sent to
the same address (eg, if you pay yourself, and also get change
back). Also remove the key amounts lists and return parameters
since we don't actually generate random ones, so we don't need
to save them as we can recalculate them when needed if we have
the correct keys.
This plugs a privacy leak, where the wallet tells the daemon
which transactions contain outputs for the wallet by asking
for additional information for that particular transaction.
As a nice bonus, this actually makes refresh slightly faster.
With RCT, we allow 0 size outputs, to try and encourage txes
with two inputs and two outputs. Consolidation would then
have two non zero inputs, one zero output, and one larger
output.
Before the normal selection, we attempt to find either one or two
suitable outputs to use as inputs to the rct tx. The intent is that
most rct txes will have one or two inputs, and we want all to look
the same if possible.
When two outputs are needed, we try to find a pair which are not
related (ie, by being from the same or similar block height).
The "transfer" simplewallet command is renamed to "transfer_original".
"transfer_new" is renamed "transfer", "transfer_rct" is removed,
and the new "transfer" now selects rct or non rct transactions
based on the current block height.
The mixRing (output keys and commitments) and II fields (key images)
can be reconstructed from vin data.
This saves some modest amount of space in the tx.
If the blockchain gets reorganized, all outputs spent in the part
of the blockchain that's blown away need to be reset to unspent
(they may end up spent again on the blocks that replace the blocks
that are removed, however).
This plugs a privacy leak from the wallet to the daemon,
as the daemon could previously see what input is included
as a transaction input, which the daemon hadn't previously
supplied. Now, the wallet requests a particular set of
outputs, including the real one.
This can result in transactions that can't be accepted if
the wallet happens to select too many outputs with non standard
unlock times. The daemon could know this and select another
output, but the wallet is blind to it. It's currently very
unlikely since I don't think anything uses non default
unlock times. The wallet requests more outputs than necessary
so it can use spares if any of the returns outputs are still
locked. If there are not enough spares to reach the desired
mixin, the transaction will fail.
This constrains the number of instances of any amount
to the unlocked ones (as defined by the default unlock time
setting: outputs with non default unlock time are not
considered, so may be counted as unlocked even if they are
not actually unlocked).
They are used to export a signed set of key images from a wallet
with a private spend key, so an auditor with the matching view key
may see which of those are spent, and which are not.
Signing is done using the spend key, since the view key may
be shared. This could be extended later, to let the user choose
which key (even a per tx key).
simplewallet's sign/verify API uses a file. The RPC uses a
string (simplewallet can't easily do strings since commands
receive a tokenized set of arguments).
Fee can now be multiplied by 2 or 3, if users want to give
priority to their transactions. There are only three levels
to avoid too much fingerprinting. Default is 1 (minimum fee).
The default multiplier can be set by "set fee-multiplier X".
It sets the max number of threads to use for a parallel job.
This is different that the number of total threads, since monero
binaries typically start a lot of them.
This sends all outputs in a wallet to a given address, alleviating
the difficulty people have had trying to send all monero but
being left with some small amount left.
modified: src/wallet/wallet2.cpp
modified: src/wallet/wallet2.h
Update to fix unconfirmed balance and give a slightly more verbose and informative confirmation message for transfers
When m_refresh_from_block_height has been set, only hashes will be
retrieved up to that height, instead of full blocks. The same will
be done for "refresh <height>" when the specified height is beyond
the current local blockchain.
b4eada9 wallet: make load_keys check types when loading JSON (moneromooo-monero)
3e55725 wallet: make the JSON reading type safe (moneromooo-monero)
f8d05f3 common: new json_util.h (moneromooo-monero)
This allows appropriate action to be taken, like displaying
the reason to the user.
Do just that in simplewallet, which should help a lot in
determining why users fail to send.
Also make it so a tx which is accepted but not relayed is
seen as a success rather than a failure.
With the change in mixin rules for v2, the "annoying" outputs are
slightly changed. There is high correlation between dust and
unmixable, but no equivalence.
It takes a filename containing JSON data to generate a wallet.
The following fields are valid:
version: integer, should be 1
filename: string, path/filename for the newly created wallet
scan_from_height: 64 bit unsigned integer, optional
password: string, optional
viewkey: string, hex representation
spendkey: string, hex representation
seed: string, optional, list of words separated by spaces
Either seed or private keys should be given. If using private
keys, the spend key may be omitted (the wallet will not be
able to spend, but will see incoming transactions).
If scan_from_height is given, blocks below this height will not
be checked for transactions as an optimization.
If it is, it points to reuse of a tx key, which isn't meant to happen.
If it does, a key image collision means that only one of those
outputs is spendable, so the wallet selects the larger amount,
unless that output was spent already.
This causes a discrepancy betewen reported received inputs and
payment total.
Since tx keys are 256 bits, this should never happen except if
done on purpose, or if a sender uses a bad PRNG.
7fc6fa3 wallet: forbid dust altogether in output selection where appropriate (moneromooo-monero)
5e1a739 blockchain: log number of outputs available for a new tx (moneromooo-monero)
The value will be different depending on whether we've reached
the first hard fork, which allows a larger size, or not.
This fixes transactions being rejected by the daemon on mainnet
where the first hard fork is not yet active.
Blockchain hashes and key images are flushed, and blocks are
pulled anew from the daemon.
The console command is shortened to match bc_height.
This should make it a lot easier on users who are currently
told to remove this particular cache file but keep the keys
one, etc, etc.
^C while in manual refresh will cancel the refresh, since that's
often an annoying thing to have to wait for. Also, a manual refresh
command will interrupt any running background refresh and take
over, rather than wait for the background refresh to be done, and
look to be hanging.
The daemon will be polled every 90 seconds for new blocks.
It is enabled by default, and can be turned on/off with
set auto-refresh 1 and set auto-refresh 0 in the wallet.
Assume the whole of a coinbase goes to the same address (so that
if the first output isn't for us, none of it is), and only look
for payment id when we received something in the transaction.
The wallet and the daemon applied different height considerations
when selecting outputs to use. This can leak information on which
input in a ring signature is the real one.
Found and originally fixed by smooth on Aeon.
f197599 wallet: encrypt the cache file (moneromooo-monero)
98c76a3 chacha8: add a key generation variant that take a pointer and size (moneromooo-monero)
It contains private data, such as a record of transactions.
The key is derived from the view and spend secret keys.
The encryption currently is one shot, so may require a lot of
memory for large wallet caches.
This obsoletes the need for a lengthy blockchain rescan when
a transaction doesn't end up in the chain after being accepted
by the daemon, or any other reason why the wallet's idea of
spent and unspent outputs gets out of sync from the blockchain's.
Pros:
- smaller on the blockchain
- shorter integrated addresses
Cons:
- less sparseness
- less ability to embed actual information
The boolean argument to encrypt payment ids is now gone from the
RPC calls, since the decision is made based on the length of the
payment id passed.
A payment ID may be encrypted using the tx secret key and the
receiver's public view key. The receiver can decrypt it with
the tx public key and the receiver's secret view key.
Using integrated addresses now cause the payment IDs to be
encrypted. Payment IDs used manually are not encrypted by default,
but can be encrypted using the new 'encrypt_payment_id' field
in the transfer and transfer_split RPC calls. It is not possible
to use an encrypted payment ID by specifying a manual simplewallet
transfer/transfer_new command, though this is just a limitation
due to input parsing.
It should avoid a lot of the issues sending more than half the
wallet's contents due to change.
Actual output selection is still random. Changing this would
improve the matching of transaction amounts to output sizes,
but may have non obvious effects on blockchain analysis.
Mapped to the new transfer_new command in simplewallet, and
transfer uses the existing algorithm.
To use in RPC, add "new_algorithm: true" in the transfer_split
JSON command. It is not used in the transfer command.
dc4dbc1 simplewallet: allow creating a wallet from a public address and view secret key (moneromooo-monero)
6a0f61d account: allow creating an account from a public address and view secret key (moneromooo-monero)
e05a58a wallet2: fix write_watch_only_wallet comment description (moneromooo-monero)
4bf6f0d simplewallet: forbid seed commands for watch only wallets (moneromooo-monero)
The new save_watch_only saves a copy of the keys file without the
spend key. It can then be given away to be used as a normal keys
file, but with no spend ability.
Sends all the dust to your own wallet. May fail (if the fee required
is more than the dust total). May end up paying most of the dust in fees.
Unlocked dust total is now also displayed in "balance".
Based on tewinget's update.
Make OpenAlias address format independent of existing DNS functions.
Add tests.
Test:
make debug-test
cd build/debug/tests/unit_tests
# test that regular DNS functions work, including IPv4 lookups.
# also test function that converts OpenAlias address format
make && ./unit_tests --gtest_filter=DNSResolver*
# test that OpenAlias addresses like donate@getmonero.org work from
# wallet tools
make && ./unit_tests --gtest_filter=AddressFromURL.Success
simplewallet run without a wallet path argument should prompt again if
an invalid path was entered.
Validity here currently means the string isn't empty.
Allow pre-JSON wallet format to load without depending on existing bin
file.
Don't write bin file while inside keys rewrite, so bin file write
behavior here matches that of regular wallet load.
Fix for simplewallet loading a wallet with a keys file but no bin file.
- this situation previously required a user to restart simplewallet
before it would refresh its blockchain from the server.
wallet2::store_keys() and wallet2::load_keys() should only use the JSON
attribute "seed_language" when applicable. That is only for
deterministic wallets.
- store_keys() don't add JSON attribute "seed_language" if
seed_language is empty
- load_keys() don't call set_seed_language if JSON attribute
"seed_language" not present
f1eaf88 Prints seed after wallet upgrade. Removed iostream include. (Oran Juice)
70971be Doxygen comments (Oran Juice)
031ca23 Rewrites to old wallet file correctly (Oran Juice)
1f833dc Doxygen comments in (Oran Juice)
0bd88ff Writes seed language while generating wallet. Wallet open fix. (Oran Juice)
09a659e Stores seed language in wallet file. added rapidjson. Yet to test backward compatibility (Oran Juice)
