Commit Graph

66 Commits

Author SHA1 Message Date
moneromooo-monero
45e419bd5c
db: store cumulative rct output distribution in the db for speed
This gets rid of the temporary precalc cache.

Also make the RPC able to send data back in binary or JSON,
since there can be a lot of data

This bumps the LMDB database format to v3, with migration.
2018-07-13 11:37:04 +01:00
stoffu
0321d1ac79
unit_tests/hardfork: add tests for get_voting_info() 2018-05-30 09:49:58 +09:00
stoffu
98cf62cc45
hardfork: fix get_next_version() 2018-05-30 09:49:57 +09:00
stoffu
7e30eadb32
unit_tests/hardfork: add tests for check_for_height() 2018-05-30 09:49:55 +09:00
stoffu
a79fc219b7
hardfork: fix get_earliest_ideal_height_for_version() to support non-existent versions 2018-05-30 09:49:52 +09:00
luigi1111
c534fe8d19
Merge pull request #3251
b9389e5 db_lmdb: save pruned and prunable tx data separately (moneromooo-monero)
2018-05-29 17:56:26 -05:00
moneromooo-monero
b9389e582e
db_lmdb: save pruned and prunable tx data separately
This bumps DB version to 2, migration code will run for v1 DBs
2018-05-23 22:48:12 +01:00
moneromooo-monero
64f82e0481
unit_tests: fix build after get_output_didstribution signature change 2018-05-21 20:49:02 +01:00
moneromooo-monero
5f8ffca34d
speed up get_output_distribution (and precalc common case) 2018-05-21 19:51:58 +01:00
Riccardo Spagni
ff31611a32 Revert "Merge pull request #3775"
This reverts commit 20ef37bbca, reversing
changes made to 40070a661f.
2018-05-21 20:30:31 +02:00
moneromooo-monero
c1886155ed
unit_tests: fix build after get_output_didstribution signature change 2018-04-28 18:02:58 +01:00
moneromooo-monero
41f727ce42
add RPC to get a histogram of outputs of a given amount 2018-03-16 10:32:26 +00:00
xmr-eric
18216f19dd Update 2018 copyright 2018-01-26 10:03:20 -05:00
moneromooo-monero
ae8602303a
Fix exceptions not finding txpool txes when relaying 2017-12-20 13:02:53 +00:00
binaryFate
10013e9434 Protect node privacy by proper filtering in restricted-mode RPC answers
This patch allows to filter out sensitive information for queries that rely on the pool state, when running in restricted mode.
This filtering is only applied to data sent back to RPC queries. Results of inline commands typed locally in the daemon are not affected.
In practice, when running with `--restricted-rpc`:
* get_transaction_pool will list relayed transactions with the fields "last relayed time" and "received time" set to zero.
* get_transaction_pool will not list transaction that have do_not_relay set to true, and will not list key images that are used only for such transactions
* get_transaction_pool_hashes.bin will not list such transaction
* get_transaction_pool_stats will not count such transactions in any of the aggregated values that are computed

The implementation does not make filtering the default, so developers should be mindful of this if they add new RPC functionality.
Fixes #2590.
2017-11-08 13:07:07 +01:00
Howard Chu
d2d8a41c47
Use actual batch size for resize estimates
And optimize import startup:

Remember start_height position during initial count_blocks pass
to avoid having to reread entire file again to arrive at start_height
2017-09-17 22:08:40 +01:00
Howard Chu
9a859844f4
Toggle SAFE syncmode on and off automatically
If monerod is started with default sync mode, set it to SAFE after
synchronization completes. Set it back to FAST if synchronization
restarts (e.g. because another peer has a longer blockchain).

If monerod is started with an explicit sync mode, none of this
automation takes effect.
2017-08-20 16:30:28 +01:00
Howard Chu
4c7f8ac04f
DB cleanup
Hide LMDB-specific stuff behind blockchain_db.h. Nobody besides blockchain_db.cpp
should ever be including DB-specific headers any more.
2017-08-19 18:11:38 +01:00
moneromooo-monero
235df7f484
blockchain_db: add a txpool tx getter which returns existence
Avoids exception spam for the "nope, not found" case
2017-06-11 15:36:48 +01:00
Howard Chu
d17c0fc2d0
Don't copy blockchain for coinbase_tx_sum
Changed Blockchain::for_all_blocks() to for_blocks_range()
Operate on blockchain in-place instead of building a copy first.
2017-06-01 14:14:24 +01:00
moneromooo-monero
b52abd1370
Move txpool to the database
Integration could go further (ie, return_tx_to_pool calls should
not be needed anymore, possibly other things).

poolstate.bin is now obsolete.
2017-05-25 22:23:37 +01:00
kenshi84
7d07c64fe5
fix dependency: put HardFork back to cryptonote_basic, made some BlockchainDB functions virtual again to avoid missing symbols error 2017-03-10 11:22:39 +09:00
moneromooo-monero
12adb4a3f3
core: move hardfork back to cryptonote_core
should fix a cross dependency betewen cryptonote_basic and
blockchain_db
2017-02-25 16:41:35 +00:00
Riccardo Spagni
c3599fa7b9
update copyright year, fix occasional lack of newline at line end 2017-02-21 19:38:18 +02:00
Riccardo Spagni
ecbd3f0798
Merge pull request #1758
5664826a unit_tests: hardfork unit test now needs get_block_blob_from_height (moneromooo-monero)
2017-02-21 17:22:01 +02:00
moneromooo-monero
5664826a84
unit_tests: hardfork unit test now needs get_block_blob_from_height 2017-02-21 14:48:34 +00:00
Howard Chu
80749e59a8
More for PR #1724
Fix unit_tests build (get_output_key API change)
2017-02-21 13:53:13 +00:00
moneromooo-monero
0288310e3b
blockchain_db: add "raw" blobdata getters for block and transaction
This speeds up operations such as serving blocks to syncing peers
2017-02-13 21:11:37 +00:00
kenshi84
8027ce0c75 extract some basic code from libcryptonote_core into libcryptonote_basic 2017-02-08 22:45:15 +09:00
moneromooo-monero
f9293b69d8
unit_tests: fix missing return after batch_start prototype change 2017-01-21 21:51:07 +00:00
Riccardo Spagni
65e33b1bc5
Merge pull request #1506
3ff54bdd Check for correct thread before ending batch transaction (Howard Chu)
eaf8470b Must wait for previous batch to finish before starting new one (Howard Chu)
c903c554 Don't cache block height, always get from DB (Howard Chu)
eb1fb601 Tweak default db-sync-mode to fast:async:1 (Howard Chu)
0693cff9 Use batch transactions when syncing (Howard Chu)
2017-01-15 14:43:12 -05:00
Howard Chu
0693cff925
Use batch transactions when syncing
Faster throughput while avoiding corruption. I.e., makes
running with --db-sync-mode safe more tolerable.
2017-01-14 22:43:06 +00:00
moneromooo-monero
0478ac6848
blockchain: allow marking "tx not found" without an exception
This is a normal occurence in many cases, and there is no need
to spam the log with those when it is.
2017-01-07 20:52:17 +00:00
Lee Clagett
8dadf02c4f Fixed undefined behavior in TestDB 2016-11-23 17:22:35 -05:00
moneromooo-monero
4fd56a52bc
unit_tests: fix build after get_output_histogram signature change 2016-10-19 09:34:12 +01:00
moneromooo-monero
6cf8ca2a7f
core: faster find_blockchain_supplement
Since this queries block heights for blocks that may or may not
exist, queries for non existing blocks would throw an exception,
and that would slow down the loop a lot. 7 seconds to go through
a 30 hash list.

Fix this by adding an optional return block height to block_exists
and using this instead. Actual errors will still throw an
exception.

This also cuts down on log exception spam.
2016-08-31 10:03:32 +01:00
moneromooo-monero
59a66e209a
move the rct commitments to the output_amounts database
Since these are needed at the same time as the output pubkeys,
this is a whole lot faster, and takes less space. Only outputs
of 0 amount store the commitment. When reading other outputs,
a fake commitment is regenerated on the fly. This avoids having
to rewrite the database to add space for fake commitments for
existing outputs.

This code relies on two things:

- LMDB must support fixed size records per key, rather than
per database (ie, all records on key 0 are the same size, all
records for non 0 keys are same size, but records from key 0
and non 0 keys do have different sizes).

- the commitment must be directly after the rest of the data
in outkey and output_data_t.
2016-08-28 21:29:02 +01:00
moneromooo-monero
eb56d0f994
blockchain_db: add functions for adding/removing/getting rct commitments 2016-08-28 21:28:11 +01:00
moneromooo-monero
11dc091464
Fake outs set is now decided by the wallet
This plugs a privacy leak from the wallet to the daemon,
as the daemon could previously see what input is included
as a transaction input, which the daemon hadn't previously
supplied. Now, the wallet requests a particular set of
outputs, including the real one.

This can result in transactions that can't be accepted if
the wallet happens to select too many outputs with non standard
unlock times. The daemon could know this and select another
output, but the wallet is blind to it. It's currently very
unlikely since I don't think anything uses non default
unlock times. The wallet requests more outputs than necessary
so it can use spares if any of the returns outputs are still
locked. If there are not enough spares to reach the desired
mixin, the transaction will fail.
2016-08-11 14:35:27 +01:00
moneromooo-monero
d7b681cd65
remove hf_starting_height db
It's not really needed, it used to be an optimization for when
that code was not using the db and needed to recalculate things
fast on startup.
2016-07-13 21:38:34 +01:00
moneroexample
de030d99a5 fix: error: -Werror=misleading-indentation
Compilation of bitmonero on Arch with gcc 6.1 results in the following
error:

/home/mwo/bitmonero/tests/unit_tests/hardfork.cpp: In member function ‘virtual void TestDB::set_hard_fork_version(uint64_t, uint8_t)’:
/home/mwo/bitmonero/tests/unit_tests/hardfork.cpp:132:5: error: this ‘if’ clause does not guard... [-Werror=misleading-indentation]
     if (versions.size() <= height) versions.resize(height+1); versions[height] = version;

This can be fixed by simply unfolding this line into three lines.
2016-05-18 10:54:41 -04:00
Howard Chu
d7ea7d9a23 Merge branch 'performance' into master 2016-04-05 21:13:16 +01:00
Howard Chu
372acee723 Cleanup
drop obsolete remove_output()
fix get_output_key(global), fix crash in blockchain_dump
2016-04-05 21:05:24 +01:00
moneromooo-monero
d5d46e6d6d
tests: obligatory hardfork unit build fix after interface change 2016-03-26 23:44:04 +00:00
moneromooo-monero
600a3cf0c0
New RPC and daemon command to get output histogram
This is a list of existing output amounts along with the number
of outputs of that amount in the blockchain.

The daemon command takes:
- no parameters: all outputs with at least 3 instances
- one parameter: all outputs with at least that many instances
- two parameters: all outputs within that many instances

The default starts at 3 to avoid massive spamming of all dust
outputs in the blockchain, and is the current minimum mixin
requirement.

An optional vector of amounts may be passed, to request
histogram only for those outputs.
2016-03-26 21:10:43 +00:00
warptangent
dfca2ae1b5
unit_tests: fix hard fork unit test compilation
Match changed BlockchainDB function declaration.
2016-03-05 10:01:10 -08:00
moneromooo-monero
8e4c2e6ebe
unit_tests: fix hard fork unit test compilation 2016-02-08 18:44:27 +00:00
moneromooo-monero
119eb10d7c
unit_tests: fix hard fork unit tests and add a test for major too 2016-02-01 20:07:15 +00:00
Howard Chu
4cf6e2b2c3 Fix 30f92f5630
Needed to add the corresponding (dummy) method to unit test hardfork
2016-01-16 16:10:16 +00:00
moneromooo-monero
a8caab1af1
unit_tests: initialize db object in ctor, not open
open isn't actually called in those tests
2016-01-02 11:37:00 +00:00