Merge pull request #961

887db9f blockchain: testnet heights for v3, v4, and v5 (moneromooo-monero)
f24ab58 ringct: remove unused code (moneromooo-monero)
b38452b ringct: pass structure by const ref, not value (moneromooo-monero)
fd11271 ringct: use memcpy/memset instead of handwritten loop where appropriate (moneromooo-monero)
5d38206 ringct: remove spurious copies (moneromooo-monero)
16732a8 rct: faster Cryptonote/rct conversions (moneromooo-monero)
fbd7c35 wallet: fix some "may be used uninitialized" warnings (moneromooo-monero)
4a41dd4 wallet: do not generate 0 change (moneromooo-monero)
a0925e6 core: use full rct signatures if just one input (moneromooo-monero)
94fd881 rct: early out on failure on verRange (moneromooo-monero)
45349b6 wallet: do not ask for duplicate histograms (moneromooo-monero)
b951bc8 wallet: transfer_selected_rct now also selects fake outs (moneromooo-monero)
4f887de increase minimum mixin to 4 on hard fork 5 (moneromooo-monero)
0815c72 core: allow v1 txes after HF 5 when sweeping unmixable outputs (moneromooo-monero)
f782d45 tests: hard fork list must end with a 0 (moneromooo-monero)
074e602 ringct: use Cryptonote serialization to hash non prunable data (moneromooo-monero)
c3b3260 New "Halfway RingCT" outputs for coinbase transactions (moneromooo-monero)
6f526cd rct: log why verification fails (moneromooo-monero)
d4b8991 rct: serialize txnFee as varint (moneromooo-monero)
d4b62a1 rct amount key modified as per luigi1111's recommendations (moneromooo-monero)
93f5c62 rct: rework v2 txes into prunable and non prunable data (moneromooo-monero)
d93746b rct: rework the verification preparation process (moneromooo-monero)
3ab2ab3 rct: change the simple flag to a type (moneromooo-monero)
c5be4b0 rct: avoid the need for the last II element (Shen Noether)
a47ceee wallet: do not store signatures in the wallet cache (moneromooo-monero)
0263dd2 core: add some locking around pool use (moneromooo-monero)
2c9d951 wallet2: factor m_spent changes (moneromooo-monero)
1303cda wallet: always use new algorithm for RPC transfers (moneromooo-monero)
b337aea rct: do not serialize senderPk - it is not used anymore (moneromooo-monero)
e5a9a47 core_tests: fix a couple pre-rct tests using rct (moneromooo-monero)
230fca2 wallet: use the prefered rct case only when enough rct outs exist (moneromooo-monero)
c27194a wallet: do not try to use rct txes a few blocks before the fork (moneromooo-monero)
1bf0698 tx_pool: log why a transaction was rejected for version checks (moneromooo-monero)
37bdf6e change fork settings to allow pre-rct txes for one more fork cycle (moneromooo-monero)
cc85cc6 simplewallet: better check_tx_key feedback (moneromooo-monero)
9b70856 rct: make the amount key derivable by a third party with the tx key (moneromooo-monero)
cf33e1a rct: do not serialize public keys in outPk (moneromooo-monero)
83ab315 wallet2_api: zero amounts are now allowed with rct (moneromooo-monero)
096ac06 wallet2_api: update on_money_{received,spent} prototypes for rct changes (moneromooo-monero)
3cb2ede rpc: send global indices along with blocks/transacions on refresh (moneromooo-monero)
414b424 core: always use the new simple rct variant (moneromooo-monero)
ce5de8b tests: add tests for wallet output selection (moneromooo-monero)
84c82cd wallet: better tx input selection (moneromooo-monero)
1e21651 rct: use the already defined H where possible (moneromooo-monero)
07d353d wallet: handle 0 change properly (moneromooo-monero)
e81a2b2 port get_tx_key/check_tx_key to rct (moneromooo-monero)
e06faef tests: add basic tests for simple rct api (moneromooo-monero)
a4d4d61 integrate simple rct api (moneromooo-monero)
1e8d37e serialization: add override for serializing bool (moneromooo-monero)
dbb5f2d ringct: optimization/cleanup of hash functions (Shen Noether)
4fd01f2 ringct: "simple" ringct variant (Shen Noether)
37c895e wallet: rct specific output selection (moneromooo-monero)
1181c57 wallet: make sweep_all work with rct txes too (moneromooo-monero)
c2ec6d3 mixable transactions must be rct for v3 (moneromooo-monero)
1017a75 wallet: factor transfer_rct code with transfer code (moneromooo-monero)
f5465d8 Condition v2 txes on v3 hard fork (moneromooo-monero)
59a66e2 move the rct commitments to the output_amounts database (moneromooo-monero)
6d0e471 rct: add the tx prefix hash into the MLSAG (moneromooo-monero)
35dce5c ringct: fix size unit mismatch calling keccak (moneromooo-monero)
20e50ec ringct: do not serialize what can be reconstructed (moneromooo-monero)
106e3dc Add rct core tests (moneromooo-monero)
ada5275 Use the supplied hard fork version in validate_miner_transaction (moneromooo-monero)
acbe06d wallet: update spent status when an accepted tx disappears (moneromooo-monero)
089df4a wallet: reset output spent status on blockchain reorg (moneromooo-monero)
73d59f1 ringct: catch errors from ge_frombytes_vartime (moneromooo-monero)
161551e tests: test for ringct rctSig data sizes (moneromooo-monero)
359f469 ringct: add missing size check for ecdhInfo (moneromooo-monero)
229968e ringct: change asserts to return false for boolean functions (moneromooo-monero)
dc4aad7 add rct to the protocol (moneromooo-monero)
211d1db db_lmdb: update reset for recent db changes (moneromooo-monero)
dee42d6 ringct: add functions to commit to an amount (moneromooo-monero)
cc7f449 make rct tx serialization work (moneromooo-monero)
e70e8a6 crypto: error out where appropriate (moneromooo-monero)
54f7429 ringct: allow no outputs, and add tests for this and fees (moneromooo-monero)
e99904a ringct: make fee optional (moneromooo-monero)
f8c04ad ringct: txn fee stuff (Shen Noether)
66f9626 ringct: new {gen,decode}Rct APIs for convenience (moneromooo-monero)
789b2e2 ringct: add more convenience functions (moneromooo-monero)
9856443 core: link against libringct (moneromooo-monero)
4258dab core: new /getrandom_rctouts.bin binary RPC call (moneromooo-monero)
c3a2e14 ringct: add convenience functions to bridge ringct and cryptonote (moneromooo-monero)
eb56d0f blockchain_db: add functions for adding/removing/getting rct commitments (moneromooo-monero)
82072e7 ringct: restore verRange check in debug mode (moneromooo-monero)
63856ca ringct: add check for destinations/amount size being equal (moneromooo-monero)
e816a09 ringct: fix off by 1 in mixin usage (moneromooo-monero)
09c5ea4 ringct: simplify random key generation (moneromooo-monero)
53cdf4d tests: new ringct test for checking H2 values (Shen Noether)
56f6549 ringct: cosmetic fixes (Shen Noether)
55ff136 ringct: changes to hashToPointSimple to calcualte H2 values (Shen Noether)
63733b1 ringct: compare keys with bitwise equality, not crypto ops (Shen Noether)
98f4c6f ringct: fix size argument to cn_fast_hash (Shen Noether)
720ac85 tests: zero inputs/outputs are in fact supposed to be accepted (moneromooo-monero)
84948ea ringct: add a test for prooveRange being non deterministic (moneromooo-monero)
09fb9f4 Fix sc_0 to skGen in ProveRange (Shen Noether)
d37c1db ringct: add a few consts where appropriate (moneromooo-monero)
700248f tests: more ringct range proof tests (moneromooo-monero)
d02f999 rct: add serialization machinery to rct types (moneromooo-monero)
0ff8305 serialization: declare do_serialize specializations before use (moneromooo-monero)
8b135e7 Added note on generating H2 (Shen Noether)
4d639d9 Fixed missing last index H2 (Shen Noether)
9e82b69 remove original Cryptonote blockchain_storage blockchain format (moneromooo-monero)
86b4426 ringct: lock access to the PRNG (moneromooo-monero)
4d7f073 ringct: add simple input validation (moneromooo-monero)
57779ab tests: add some more ringct building block tests (moneromooo-monero)
b656001 ringct: add convenience operators to key (moneromooo-monero)
2d6303f tests: add Shen Noether's basic ringct tests (moneromooo-monero)
9b1afe5 ringct: import of Shen Noether's ring confidential transactions (moneromooo-monero)

CMakeLists.txt

@@ -235,10 +235,6 @@ elseif (DATABASE STREQUAL "berkeleydb")
 
   message(STATUS "Using Berkeley DB as default DB type")
   add_definitions("-DDEFAULT_DB_TYPE=\"berkeley\"")
-elseif (DATABASE STREQUAL "memory")
-  set(BLOCKCHAIN_DB DB_MEMORY)
-  message(STATUS "Using Serialised In Memory as default DB type")
-  add_definitions("-DDEFAULT_DB_TYPE=\"memory\"")
 else()
   die("Invalid database type: ${DATABASE}")
 endif()

src/CMakeLists.txt

@@ -91,6 +91,7 @@ endfunction ()
 
 add_subdirectory(common)
 add_subdirectory(crypto)
+add_subdirectory(ringct)
 add_subdirectory(cryptonote_core)
 add_subdirectory(blockchain_db)
 add_subdirectory(mnemonics)

src/blockchain_db/berkeleydb/db_bdb.cpp

@@ -361,7 +361,7 @@ void BlockchainBDB::remove_transaction_data(const crypto::hash& tx_hash, const t
         throw1(DB_ERROR("Failed to add removal of tx outputs to db transaction"));
 }
 
-void BlockchainBDB::add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index, const uint64_t unlock_time)
+void BlockchainBDB::add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index, const uint64_t unlock_time, const rct::key *commitment)
 {
     LOG_PRINT_L3("BlockchainBDB::" << __func__);
     check_open();

src/blockchain_db/berkeleydb/db_bdb.h

@@ -364,7 +364,7 @@ private:
 
   virtual void remove_transaction_data(const crypto::hash& tx_hash, const transaction& tx);
 
-  virtual void add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index, const uint64_t unlock_time);
+  virtual void add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index, const uint64_t unlock_time, const rct::key *commitment);
 
   virtual void remove_output(const tx_out& tx_output);
 

src/blockchain_db/blockchain_db.cpp

@@ -46,6 +46,7 @@ void BlockchainDB::pop_block()
 
 void BlockchainDB::add_transaction(const crypto::hash& blk_hash, const transaction& tx, const crypto::hash* tx_hash_ptr)
 {
+  bool miner_tx = false;
   crypto::hash tx_hash;
   if (!tx_hash_ptr)
   {
@@ -67,6 +68,7 @@ void BlockchainDB::add_transaction(const crypto::hash& blk_hash, const transacti
     else if (tx_input.type() == typeid(txin_gen))
     {
       /* nothing to do here */
+      miner_tx = true;
     }
     else
     {
@@ -90,7 +92,21 @@ void BlockchainDB::add_transaction(const crypto::hash& blk_hash, const transacti
   // we need the index
   for (uint64_t i = 0; i < tx.vout.size(); ++i)
   {
-    amount_output_indices.push_back(add_output(tx_hash, tx.vout[i], i, tx.unlock_time));
+    // miner v2 txes have their coinbase output in one single out to save space,
+    // and we store them as rct outputs with an identity mask
+    if (miner_tx && tx.version == 2)
+    {
+      cryptonote::tx_out vout = tx.vout[i];
+      rct::key commitment = rct::zeroCommit(vout.amount);
+      vout.amount = 0;
+      amount_output_indices.push_back(add_output(tx_hash, vout, i, tx.unlock_time,
+        &commitment));
+    }
+    else
+    {
+      amount_output_indices.push_back(add_output(tx_hash, tx.vout[i], i, tx.unlock_time,
+        tx.version > 1 ? &tx.rct_signatures.outPk[i].mask : NULL));
+    }
   }
   add_tx_amount_output_indices(tx_id, amount_output_indices);
 }

src/blockchain_db/blockchain_db.h

@@ -110,6 +110,7 @@ struct output_data_t
   crypto::public_key pubkey;       //!< the output's public key (for spend verification)
   uint64_t           unlock_time;  //!< the output's unlock time (or height)
   uint64_t           height;       //!< the height of the block which created the output
+  rct::key           commitment;   //!< the output's amount commitment (for spend verification)
 };
 #pragma pack(pop)
 
@@ -407,9 +408,10 @@ private:
    * @param tx_output the output
    * @param local_index index of the output in its transaction
    * @param unlock_time unlock time/height of the output
+   * @param commitment the rct commitment to the output amount
    * @return amount output index
    */
-  virtual uint64_t add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index, const uint64_t unlock_time) = 0;
+  virtual uint64_t add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index, const uint64_t unlock_time, const rct::key *commitment) = 0;
 
   /**
    * @brief store amount output indices for a tx's outputs

src/blockchain_db/lmdb/db_lmdb.cpp

@@ -51,6 +51,13 @@ using epee::string_tools::pod_to_hex;
 namespace
 {
 
+struct pre_rct_output_data_t
+{
+  crypto::public_key pubkey;       //!< the output's public key (for spend verification)
+  uint64_t           unlock_time;  //!< the output's unlock time (or height)
+  uint64_t           height;       //!< the height of the block which created the output
+};
+
 template <typename T>
 inline void throw0(const T &e)
 {
@@ -242,6 +249,12 @@ typedef struct txindex {
     tx_data_t data;
 } txindex;
 
+typedef struct pre_rct_outkey {
+    uint64_t amount_index;
+    uint64_t output_id;
+    pre_rct_output_data_t data;
+} pre_rct_outkey;
+
 typedef struct outkey {
     uint64_t amount_index;
     uint64_t output_id;
@@ -765,7 +778,8 @@ void BlockchainLMDB::remove_transaction_data(const crypto::hash& tx_hash, const
 uint64_t BlockchainLMDB::add_output(const crypto::hash& tx_hash,
     const tx_out& tx_output,
     const uint64_t& local_index,
-    const uint64_t unlock_time)
+    const uint64_t unlock_time,
+    const rct::key *commitment)
 {
   LOG_PRINT_L3("BlockchainLMDB::" << __func__);
   check_open();
@@ -778,6 +792,8 @@ uint64_t BlockchainLMDB::add_output(const crypto::hash& tx_hash,
 
   if (tx_output.target.type() != typeid(txout_to_key))
     throw0(DB_ERROR("Wrong output type: expected txout_to_key"));
+  if (tx_output.amount == 0 && !commitment)
+    throw0(DB_ERROR("RCT output without commitment"));
 
   outtx ot = {m_num_outputs, tx_hash, local_index};
   MDB_val_set(vot, ot);
@@ -806,8 +822,16 @@ uint64_t BlockchainLMDB::add_output(const crypto::hash& tx_hash,
   ok.data.pubkey = boost::get < txout_to_key > (tx_output.target).key;
   ok.data.unlock_time = unlock_time;
   ok.data.height = m_height;
-  data.mv_data = &ok;
+  if (tx_output.amount == 0)
+  {
+    ok.data.commitment = *commitment;
     data.mv_size = sizeof(ok);
+  }
+  else
+  {
+    data.mv_size = sizeof(pre_rct_outkey);
+  }
+  data.mv_data = &ok;
 
   if ((result = mdb_cursor_put(m_cur_output_amounts, &val_amount, &data, MDB_APPENDDUP)))
       throw0(DB_ERROR(lmdb_error("Failed to add output pubkey to db transaction: ", result).c_str()));
@@ -1234,6 +1258,7 @@ void BlockchainLMDB::reset()
   mdb_txn_safe txn;
   if (auto result = mdb_txn_begin(m_env, NULL, 0, txn))
     throw0(DB_ERROR(lmdb_error("Failed to create a transaction for the db: ", result).c_str()));
+
   if (auto result = mdb_drop(txn, m_blocks, 0))
     throw0(DB_ERROR(lmdb_error("Failed to drop m_blocks: ", result).c_str()));
   if (auto result = mdb_drop(txn, m_block_info, 0))
@@ -1242,6 +1267,8 @@ void BlockchainLMDB::reset()
     throw0(DB_ERROR(lmdb_error("Failed to drop m_block_heights: ", result).c_str()));
   if (auto result = mdb_drop(txn, m_txs, 0))
     throw0(DB_ERROR(lmdb_error("Failed to drop m_txs: ", result).c_str()));
+  if (auto result = mdb_drop(txn, m_tx_indices, 0))
+    throw0(DB_ERROR(lmdb_error("Failed to drop m_tx_indices: ", result).c_str()));
   if (auto result = mdb_drop(txn, m_tx_outputs, 0))
     throw0(DB_ERROR(lmdb_error("Failed to drop m_tx_outputs: ", result).c_str()));
   if (auto result = mdb_drop(txn, m_output_txs, 0))
@@ -1255,6 +1282,13 @@ void BlockchainLMDB::reset()
     throw0(DB_ERROR(lmdb_error("Failed to drop m_hf_versions: ", result).c_str()));
   if (auto result = mdb_drop(txn, m_properties, 0))
     throw0(DB_ERROR(lmdb_error("Failed to drop m_properties: ", result).c_str()));
+
+  // init with current version
+  MDB_val_copy<const char*> k("version");
+  MDB_val_copy<uint32_t> v(VERSION);
+  if (auto result = mdb_put(txn, m_properties, &k, &v, 0))
+    throw0(DB_ERROR(lmdb_error("Failed to write version to database: ", result).c_str()));
+
   txn.commit();
   m_height = 0;
   m_num_outputs = 0;
@@ -1923,8 +1957,19 @@ output_data_t BlockchainLMDB::get_output_key(const uint64_t& amount, const uint6
     throw1(OUTPUT_DNE("Attempting to get output pubkey by index, but key does not exist"));
   else if (get_result)
     throw0(DB_ERROR("Error attempting to retrieve an output pubkey from the db"));
-  outkey *okp = (outkey *)v.mv_data;
+
-  output_data_t ret = okp->data;
+  output_data_t ret;
+  if (amount == 0)
+  {
+    const outkey *okp = (const outkey *)v.mv_data;
+    ret = okp->data;
+  }
+  else
+  {
+    const pre_rct_outkey *okp = (const pre_rct_outkey *)v.mv_data;
+    memcpy(&ret, &okp->data, sizeof(pre_rct_output_data_t));;
+    ret.commitment = rct::zeroCommit(amount);
+  }
   TXN_POSTFIX_RDONLY();
   return ret;
 }
@@ -2547,8 +2592,18 @@ void BlockchainLMDB::get_output_key(const uint64_t &amount, const std::vector<ui
     else if (get_result)
       throw0(DB_ERROR(lmdb_error("Error attempting to retrieve an output pubkey from the db", get_result).c_str()));
 
-    outkey *okp = (outkey *)v.mv_data;
+    output_data_t data;
-    output_data_t data = okp->data;
+    if (amount == 0)
+    {
+      const outkey *okp = (const outkey *)v.mv_data;
+      data = okp->data;
+    }
+    else
+    {
+      const pre_rct_outkey *okp = (const pre_rct_outkey *)v.mv_data;
+      memcpy(&data, &okp->data, sizeof(pre_rct_output_data_t));
+      data.commitment = rct::zeroCommit(amount);
+    }
     outputs.push_back(data);
   }
 

src/blockchain_db/lmdb/db_lmdb.h

@@ -30,6 +30,7 @@
 
 #include "blockchain_db/blockchain_db.h"
 #include "cryptonote_protocol/blobdatatype.h" // for type blobdata
+#include "ringct/rctTypes.h"
 #include <boost/thread/tss.hpp>
 
 #include <lmdb.h>
@@ -292,7 +293,8 @@ private:
   virtual uint64_t add_output(const crypto::hash& tx_hash,
       const tx_out& tx_output,
       const uint64_t& local_index,
-      const uint64_t unlock_time
+      const uint64_t unlock_time,
+      const rct::key *commitment
       );
 
   virtual void add_tx_amount_output_indices(const uint64_t tx_id,

src/blockchain_utilities/CMakeLists.txt

@@ -26,15 +26,6 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
-set(blockchain_converter_sources
-  blockchain_converter.cpp
-  )
-
-set(blockchain_converter_private_headers)
-
-bitmonero_private_headers(blockchain_converter
-	  ${blockchain_converter_private_headers})
-
 set(blockchain_import_sources
   blockchain_import.cpp
   bootstrap_file.cpp
@@ -87,30 +78,6 @@ bitmonero_private_headers(cn_deserialize
 	  ${cn_deserialize_private_headers})
 
 
-if (BLOCKCHAIN_DB STREQUAL DB_LMDB)
-bitmonero_add_executable(blockchain_converter
-  ${blockchain_converter_sources}
-  ${blockchain_converter_private_headers})
-
-target_link_libraries(blockchain_converter
-  LINK_PRIVATE
-    cryptonote_core
-	p2p
-	blockchain_db
-    ${CMAKE_THREAD_LIBS_INIT})
-
-if(ARCH_WIDTH)
-  target_compile_definitions(blockchain_converter
-    PUBLIC -DARCH_WIDTH=${ARCH_WIDTH})
-endif()
-
-add_dependencies(blockchain_converter
-	version)
-set_property(TARGET blockchain_converter
-	PROPERTY
-	OUTPUT_NAME "blockchain_converter")
-endif ()
-
 bitmonero_add_executable(blockchain_import
   ${blockchain_import_sources}
   ${blockchain_import_private_headers})

src/blockchain_utilities/README.md

@@ -18,9 +18,10 @@ e.g.
 
 This is also the default compile setting on the master branch.
 
-By default, the exporter will use the original in-memory database (blockchain.bin) as its source.
+The exporter will use the LMDB database as its source.
-This default is to make migrating to an LMDB database easy, without having to recompile anything.
+If you are still using an old style in-memory database (blockchain.bin), you will
-To change the source, adjust `SOURCE_DB` in `src/blockchain_utilities/bootstrap_file.h` according to the comments.
+have to either resync from scratch, or use an older version of the tools to export
+it and import it.
 
 ## Usage:
 

src/blockchain_utilities/blockchain_converter.cpp

@@ -1,306 +0,0 @@
-// Copyright (c) 2014-2016, The Monero Project
-//
-// All rights reserved.
-//
-// Redistribution and use in source and binary forms, with or without modification, are
-// permitted provided that the following conditions are met:
-//
-// 1. Redistributions of source code must retain the above copyright notice, this list of
-//    conditions and the following disclaimer.
-//
-// 2. Redistributions in binary form must reproduce the above copyright notice, this list
-//    of conditions and the following disclaimer in the documentation and/or other
-//    materials provided with the distribution.
-//
-// 3. Neither the name of the copyright holder nor the names of its contributors may be
-//    used to endorse or promote products derived from this software without specific
-//    prior written permission.
-//
-// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
-// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
-// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
-// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
-// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
-// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-#include "include_base_utils.h"
-#include "common/util.h"
-#include "warnings.h"
-#include "crypto/crypto.h"
-#include "cryptonote_config.h"
-#include "cryptonote_core/cryptonote_format_utils.h"
-#include "misc_language.h"
-#include "cryptonote_core/blockchain_storage.h"
-#include "blockchain_db/blockchain_db.h"
-#include "cryptonote_core/blockchain.h"
-#include "blockchain_db/lmdb/db_lmdb.h"
-#include "cryptonote_core/tx_pool.h"
-#include "common/command_line.h"
-#include "serialization/json_utils.h"
-#include "include_base_utils.h"
-#include "version.h"
-#include <iostream>
-
-
-namespace
-{
-
-// CONFIG
-bool opt_batch   = true;
-bool opt_resume  = true;
-bool opt_testnet = false;
-
-// number of blocks per batch transaction
-// adjustable through command-line argument according to available RAM
-#if ARCH_WIDTH != 32
-uint64_t db_batch_size_verify = 5000;
-#else
-// set a lower default batch size for Windows, pending possible LMDB issue with
-// large batch size.
-uint64_t db_batch_size_verify = 100;
-#endif
-
-// converter only uses verify mode
-uint64_t db_batch_size = db_batch_size_verify;
-
-}
-
-namespace po = boost::program_options;
-
-using namespace cryptonote;
-using namespace epee;
-
-struct fake_core
-{
-  Blockchain dummy;
-  tx_memory_pool m_pool;
-
-  blockchain_storage m_storage;
-
-#if !defined(BLOCKCHAIN_DB)
-  // for multi_db_runtime:
-  fake_core(const boost::filesystem::path &path, const bool use_testnet) : dummy(m_pool), m_pool(&dummy), m_storage(m_pool)
-#else
-  // for multi_db_compile:
-  fake_core(const boost::filesystem::path &path, const bool use_testnet) : dummy(m_pool), m_pool(dummy), m_storage(&m_pool)
-#endif
-  {
-    m_pool.init(path.string());
-    m_storage.init(path.string(), use_testnet);
-  }
-};
-
-int main(int argc, char* argv[])
-{
-  uint64_t height = 0;
-  uint64_t start_block = 0;
-  uint64_t end_block = 0;
-  uint64_t num_blocks = 0;
-
-  boost::filesystem::path default_data_path {tools::get_default_data_dir()};
-  boost::filesystem::path default_testnet_data_path {default_data_path / "testnet"};
-
-
-  po::options_description desc_cmd_only("Command line options");
-  po::options_description desc_cmd_sett("Command line options and settings options");
-  const command_line::arg_descriptor<uint32_t> arg_log_level   =  {"log-level", "", LOG_LEVEL_0};
-  const command_line::arg_descriptor<uint64_t> arg_batch_size  =  {"batch-size", "", db_batch_size};
-  const command_line::arg_descriptor<bool>     arg_testnet_on  = {
-    "testnet"
-      , "Run on testnet."
-      , opt_testnet
-  };
-  const command_line::arg_descriptor<uint64_t> arg_block_number =
-  {"block-number", "Number of blocks (default: use entire source blockchain)",
-    0};
-
-  command_line::add_arg(desc_cmd_sett, command_line::arg_data_dir, default_data_path.string());
-  command_line::add_arg(desc_cmd_sett, command_line::arg_testnet_data_dir, default_testnet_data_path.string());
-  command_line::add_arg(desc_cmd_sett, arg_log_level);
-  command_line::add_arg(desc_cmd_sett, arg_batch_size);
-  command_line::add_arg(desc_cmd_sett, arg_testnet_on);
-  command_line::add_arg(desc_cmd_sett, arg_block_number);
-
-  command_line::add_arg(desc_cmd_only, command_line::arg_help);
-
-  const command_line::arg_descriptor<bool> arg_batch  =  {"batch",
-    "Batch transactions for faster import", true};
-  const command_line::arg_descriptor<bool> arg_resume =  {"resume",
-        "Resume from current height if output database already exists", true};
-
-  // call add_options() directly for these arguments since command_line helpers
-  // support only boolean switch, not boolean argument
-  desc_cmd_sett.add_options()
-    (arg_batch.name,  make_semantic(arg_batch),  arg_batch.description)
-    (arg_resume.name, make_semantic(arg_resume), arg_resume.description)
-    ;
-
-  po::options_description desc_options("Allowed options");
-  desc_options.add(desc_cmd_only).add(desc_cmd_sett);
-
-
-  po::variables_map vm;
-  bool r = command_line::handle_error_helper(desc_options, [&]()
-  {
-    po::store(po::parse_command_line(argc, argv, desc_options), vm);
-    po::notify(vm);
-
-    return true;
-  });
-  if (!r)
-    return 1;
-
-  int log_level = command_line::get_arg(vm, arg_log_level);
-  opt_batch     = command_line::get_arg(vm, arg_batch);
-  opt_resume    = command_line::get_arg(vm, arg_resume);
-  db_batch_size = command_line::get_arg(vm, arg_batch_size);
-
-  if (command_line::get_arg(vm, command_line::arg_help))
-  {
-    std::cout << "Monero '" << MONERO_RELEASE_NAME << "' (v" << MONERO_VERSION_FULL << ")" << ENDL << ENDL;
-    std::cout << desc_options << std::endl;
-    return 1;
-  }
-
-  if (! opt_batch && ! vm["batch-size"].defaulted())
-  {
-    std::cerr << "Error: batch-size set, but batch option not enabled" << ENDL;
-    return 1;
-  }
-  if (! db_batch_size)
-  {
-    std::cerr << "Error: batch-size must be > 0" << ENDL;
-    return 1;
-  }
-
-  log_space::get_set_log_detalisation_level(true, log_level);
-  log_space::log_singletone::add_logger(LOGGER_CONSOLE, NULL, NULL);
-  LOG_PRINT_L0("Starting...");
-
-  std::string src_folder;
-  opt_testnet = command_line::get_arg(vm, arg_testnet_on);
-  auto data_dir_arg = opt_testnet ? command_line::arg_testnet_data_dir : command_line::arg_data_dir;
-  src_folder = command_line::get_arg(vm, data_dir_arg);
-  boost::filesystem::path dest_folder(src_folder);
-
-  num_blocks = command_line::get_arg(vm, arg_block_number);
-
-  if (opt_batch)
-  {
-    LOG_PRINT_L0("batch:   " << std::boolalpha << opt_batch << std::noboolalpha
-        << "  batch size: " << db_batch_size);
-  }
-  else
-  {
-    LOG_PRINT_L0("batch:   " << std::boolalpha << opt_batch << std::noboolalpha);
-  }
-  LOG_PRINT_L0("resume:  " << std::boolalpha << opt_resume  << std::noboolalpha);
-  LOG_PRINT_L0("testnet: " << std::boolalpha << opt_testnet << std::noboolalpha);
-
-  fake_core c(src_folder, opt_testnet);
-
-  height = c.m_storage.get_current_blockchain_height();
-
-  BlockchainDB *blockchain;
-  blockchain = new BlockchainLMDB(opt_batch);
-  dest_folder /= blockchain->get_db_name();
-  LOG_PRINT_L0("Source blockchain: " << src_folder);
-  LOG_PRINT_L0("Dest blockchain:   " << dest_folder.string());
-  LOG_PRINT_L0("Opening dest blockchain (BlockchainDB " << blockchain->get_db_name() << ")");
-  blockchain->open(dest_folder.string());
-  LOG_PRINT_L0("Source blockchain height: " << height);
-  LOG_PRINT_L0("Dest blockchain height:   " << blockchain->height());
-
-  if (opt_resume)
-    // next block number to add is same as current height
-    start_block = blockchain->height();
-
-  if (! num_blocks || (start_block + num_blocks > height))
-    end_block = height - 1;
-  else
-    end_block = start_block + num_blocks - 1;
-
-  LOG_PRINT_L0("start height: " << start_block+1 << "  stop height: " <<
-            end_block+1);
-
-  if (start_block > end_block)
-  {
-    LOG_PRINT_L0("Finished: no blocks to add");
-    delete blockchain;
-    return 0;
-  }
-
-  if (opt_batch)
-    blockchain->batch_start(db_batch_size);
-  uint64_t i = 0;
-  for (i = start_block; i < end_block + 1; ++i)
-  {
-    // block: i  height: i+1  end height: end_block + 1
-    if ((i+1) % 10 == 0)
-    {
-      std::cout << "\r                   \r" << "height " << i+1 << "/" <<
-        end_block+1 << " (" << (i+1)*100/(end_block+1)<< "%)" << std::flush;
-    }
-    // for debugging:
-    // std::cout << "height " << i+1 << "/" << end_block+1
-    //   << " ((" << i+1 << ")*100/(end_block+1))" << "%)" << ENDL;
-
-    block b = c.m_storage.get_block(i);
-    size_t bsize = c.m_storage.get_block_size(i);
-    difficulty_type bdiff = c.m_storage.get_block_cumulative_difficulty(i);
-    uint64_t bcoins = c.m_storage.get_block_coins_generated(i);
-    std::vector<transaction> txs;
-    std::vector<crypto::hash> missed;
-
-    c.m_storage.get_transactions(b.tx_hashes, txs, missed);
-    if (missed.size())
-    {
-      std::cout << ENDL;
-      std::cerr << "Missed transaction(s) for block at height " << i + 1 << ", exiting" << ENDL;
-      delete blockchain;
-      return 1;
-    }
-
-    try
-    {
-      blockchain->add_block(b, bsize, bdiff, bcoins, txs);
-
-      if (opt_batch)
-      {
-        if ((i < end_block) && ((i + 1) % db_batch_size == 0))
-        {
-          std::cout << "\r                   \r";
-          std::cout << "[- batch commit at height " << i + 1 << " -]" << ENDL;
-          blockchain->batch_stop();
-          blockchain->batch_start(db_batch_size);
-          std::cout << ENDL;
-          blockchain->show_stats();
-        }
-      }
-    }
-    catch (const std::exception& e)
-    {
-      std::cout << ENDL;
-      std::cerr << "Error adding block " << i << " to new blockchain: " << e.what() << ENDL;
-      delete blockchain;
-      return 2;
-    }
-  }
-  if (opt_batch)
-  {
-    std::cout << "\r                   \r" << "height " << i << "/" <<
-      end_block+1 << " (" << (i)*100/(end_block+1)<< "%)" << std::flush;
-    std::cout << ENDL;
-    std::cout << "[- batch commit at height " << i << " -]" << ENDL;
-    blockchain->batch_stop();
-  }
-  std::cout << ENDL;
-  blockchain->show_stats();
-  std::cout << "Finished at height: " << i << "  block: " << i-1 << ENDL;
-
-  delete blockchain;
-  return 0;
-}

src/blockchain_utilities/blockchain_dump.cpp

@@ -27,8 +27,8 @@
 // THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 #include "cryptonote_core/cryptonote_basic.h"
-#include "cryptonote_core/blockchain_storage.h"
 #include "cryptonote_core/blockchain.h"
+#include "cryptonote_core/tx_pool.h"
 #include "blockchain_db/blockchain_db.h"
 #include "blockchain_db/lmdb/db_lmdb.h"
 #ifdef BERKELEY_DB
@@ -103,7 +103,6 @@ int main(int argc, char* argv[])
   const command_line::arg_descriptor<std::string> arg_output_file = {"output-file", "Specify output file", "", true};
   const command_line::arg_descriptor<uint32_t> arg_log_level  = {"log-level",  "", log_level};
   const command_line::arg_descriptor<uint64_t> arg_block_stop = {"block-stop", "Stop at block number", block_stop};
-#if SOURCE_DB != DB_MEMORY
   const command_line::arg_descriptor<std::string> arg_db_type = {
     "db-type"
   , "Specify database type"
@@ -114,7 +113,6 @@ int main(int argc, char* argv[])
       , "Include data that is only in a database version."
       , false
   };
-#endif
   const command_line::arg_descriptor<bool>     arg_testnet_on = {
     "testnet"
       , "Run on testnet."
@@ -125,10 +123,8 @@ int main(int argc, char* argv[])
   command_line::add_arg(desc_cmd_sett, command_line::arg_data_dir, default_data_path.string());
   command_line::add_arg(desc_cmd_sett, command_line::arg_testnet_data_dir, default_testnet_data_path.string());
   command_line::add_arg(desc_cmd_sett, arg_output_file);
-#if SOURCE_DB != DB_MEMORY
   command_line::add_arg(desc_cmd_sett, arg_db_type);
   command_line::add_arg(desc_cmd_sett, arg_include_db_only_data);
-#endif
   command_line::add_arg(desc_cmd_sett, arg_testnet_on);
   command_line::add_arg(desc_cmd_sett, arg_log_level);
   command_line::add_arg(desc_cmd_sett, arg_block_stop);
@@ -164,9 +160,7 @@ int main(int argc, char* argv[])
   LOG_PRINT_L0("Setting log level = " << log_level);
 
   bool opt_testnet = command_line::get_arg(vm, arg_testnet_on);
-#if SOURCE_DB != DB_MEMORY
   bool opt_include_db_only_data = command_line::get_arg(vm, arg_include_db_only_data);
-#endif
 
   std::string m_config_folder;
 
@@ -208,15 +202,6 @@ int main(int argc, char* argv[])
 
   // If we wanted to use the memory pool, we would set up a fake_core.
 
-#if SOURCE_DB == DB_MEMORY
-  // blockchain_storage* core_storage = NULL;
-  // tx_memory_pool m_mempool(*core_storage); // is this fake anyway? just passing in NULL! so m_mempool can't be used anyway, right?
-  // core_storage = new blockchain_storage(&m_mempool);
-
-  blockchain_storage* core_storage = new blockchain_storage(NULL);
-  LOG_PRINT_L0("Initializing source blockchain (in-memory database)");
-  r = core_storage->init(m_config_folder, opt_testnet);
-#else
   // Use Blockchain instead of lower-level BlockchainDB for two reasons:
   // 1. Blockchain has the init() method for easy setup
   // 2. exporter needs to use get_current_blockchain_height(), get_block_id_by_height(), get_block_by_hash()
@@ -267,7 +252,6 @@ int main(int argc, char* argv[])
     return 1;
   }
   r = core_storage->init(db, opt_testnet);
-#endif
 
   CHECK_AND_ASSERT_MES(r, false, "Failed to initialize source blockchain storage");
   LOG_PRINT_L0("Source blockchain storage initialized OK");
@@ -340,7 +324,6 @@ int main(int argc, char* argv[])
         write_pod(d,key_images[n]);
     }
     end_compound(d);
-#if SOURCE_DB != DB_MEMORY
     if (opt_include_db_only_data)
     {
       start_struct(d, "block_timestamps", true);
@@ -420,7 +403,6 @@ int main(int argc, char* argv[])
           write_pod(d, boost::lexical_cast<std::string>(h), (unsigned int)db->get_hard_fork_version(h));
       end_compound(d);
     }
-#endif
   end_compound(d);
 
   CHECK_AND_ASSERT_MES(r, false, "Failed to dump blockchain");

src/blockchain_utilities/blockchain_export.cpp

@@ -29,6 +29,7 @@
 #include "bootstrap_file.h"
 #include "blocksdat_file.h"
 #include "common/command_line.h"
+#include "cryptonote_core/tx_pool.h"
 #include "blockchain_db/blockchain_db.h"
 #include "blockchain_db/lmdb/db_lmdb.h"
 #if defined(BERKELEY_DB)
@@ -53,11 +54,7 @@ std::string join_set_strings(const std::unordered_set<std::string>& db_types_all
 
 int main(int argc, char* argv[])
 {
-#if defined(BLOCKCHAIN_DB) && (BLOCKCHAIN_DB == DB_MEMORY)
-  std::string default_db_type = "memory";
-#else
   std::string default_db_type = "lmdb";
-#endif
 
   std::unordered_set<std::string> db_types_all = cryptonote::blockchain_db_types;
   db_types_all.insert("memory");
@@ -160,15 +157,6 @@ int main(int argc, char* argv[])
 
   // If we wanted to use the memory pool, we would set up a fake_core.
 
-#if SOURCE_DB == DB_MEMORY
-  // blockchain_storage* core_storage = NULL;
-  // tx_memory_pool m_mempool(*core_storage); // is this fake anyway? just passing in NULL! so m_mempool can't be used anyway, right?
-  // core_storage = new blockchain_storage(&m_mempool);
-
-  blockchain_storage* core_storage = new blockchain_storage(NULL);
-  LOG_PRINT_L0("Initializing source blockchain (in-memory database)");
-  r = core_storage->init(m_config_folder, opt_testnet);
-#else
   // Use Blockchain instead of lower-level BlockchainDB for two reasons:
   // 1. Blockchain has the init() method for easy setup
   // 2. exporter needs to use get_current_blockchain_height(), get_block_id_by_height(), get_block_by_hash()
@@ -217,7 +205,6 @@ int main(int argc, char* argv[])
     return 1;
   }
   r = core_storage->init(db, opt_testnet);
-#endif
 
   CHECK_AND_ASSERT_MES(r, false, "Failed to initialize source blockchain storage");
   LOG_PRINT_L0("Source blockchain storage initialized OK");

src/blockchain_utilities/blockchain_import.cpp

@@ -222,12 +222,8 @@ int pop_blocks(FakeCore& simple_core, int num_blocks)
   std::vector<transaction> popped_txs;
   for (int i=0; i < num_blocks; ++i)
   {
-#if defined(BLOCKCHAIN_DB) && (BLOCKCHAIN_DB == DB_MEMORY)
-    simple_core.m_storage.debug_pop_block_from_blockchain();
-#else
     // simple_core.m_storage.pop_block_from_blockchain() is private, so call directly through db
     simple_core.m_storage.get_db().pop_block(popped_block, popped_txs);
-#endif
     quit = 1;
   }
 
@@ -244,9 +240,7 @@ int pop_blocks(FakeCore& simple_core, int num_blocks)
     {
       simple_core.batch_stop();
     }
-#if !defined(BLOCKCHAIN_DB) || (BLOCKCHAIN_DB == DB_LMDB)
     simple_core.m_storage.get_db().show_stats();
-#endif
   }
 
   return num_blocks;
@@ -255,11 +249,6 @@ int pop_blocks(FakeCore& simple_core, int num_blocks)
 template <typename FakeCore>
 int import_from_file(FakeCore& simple_core, const std::string& import_file_path, uint64_t block_stop=0)
 {
-#if !defined(BLOCKCHAIN_DB)
-  static_assert(std::is_same<fake_core_memory, FakeCore>::value || std::is_same<fake_core_db, FakeCore>::value,
-      "FakeCore constraint error");
-#endif
-#if !defined(BLOCKCHAIN_DB) || (BLOCKCHAIN_DB == DB_LMDB)
   if (std::is_same<fake_core_db, FakeCore>::value)
   {
     // Reset stats, in case we're using newly created db, accumulating stats
@@ -267,7 +256,6 @@ int import_from_file(FakeCore& simple_core, const std::string& import_file_path,
     // This aligns internal db counts with importer counts.
     simple_core.m_storage.get_db().reset_stats();
   }
-#endif
   boost::filesystem::path fs_import_file_path(import_file_path);
   boost::system::error_code ec;
   if (!boost::filesystem::exists(fs_import_file_path, ec))
@@ -567,9 +555,7 @@ int import_from_file(FakeCore& simple_core, const std::string& import_file_path,
             simple_core.batch_stop();
             simple_core.batch_start(db_batch_size);
             std::cout << ENDL;
-#if !defined(BLOCKCHAIN_DB) || (BLOCKCHAIN_DB == DB_LMDB)
             simple_core.m_storage.get_db().show_stats();
-#endif
           }
         }
       }
@@ -595,9 +581,7 @@ int import_from_file(FakeCore& simple_core, const std::string& import_file_path,
     {
       simple_core.batch_stop();
     }
-#if !defined(BLOCKCHAIN_DB) || (BLOCKCHAIN_DB == DB_LMDB)
     simple_core.m_storage.get_db().show_stats();
-#endif
     LOG_PRINT_L0("Number of blocks imported: " << num_imported);
     if (h > 0)
       // TODO: if there was an error, the last added block is probably at zero-based height h-2
@@ -609,13 +593,8 @@ int import_from_file(FakeCore& simple_core, const std::string& import_file_path,
 
 int main(int argc, char* argv[])
 {
-#if defined(BLOCKCHAIN_DB) && (BLOCKCHAIN_DB == DB_MEMORY)
-  std::string default_db_type = "memory";
-  std::string default_db_engine_compiled = "memory";
-#else
   std::string default_db_type = "lmdb";
   std::string default_db_engine_compiled = "blockchain_db";
-#endif
 
   std::unordered_set<std::string> db_types_all = cryptonote::blockchain_db_types;
   db_types_all.insert("memory");
@@ -817,17 +796,11 @@ int main(int argc, char* argv[])
   // circumstance.
 
   // for multi_db_runtime:
-#if !defined(BLOCKCHAIN_DB)
   if (db_type == "lmdb" || db_type == "berkeley")
   {
     fake_core_db simple_core(m_config_folder, opt_testnet, opt_batch, db_type, db_flags);
     import_from_file(simple_core, import_file_path, block_stop);
   }
-  else if (db_type == "memory")
-  {
-    fake_core_memory simple_core(m_config_folder, opt_testnet);
-    import_from_file(simple_core, import_file_path, block_stop);
-  }
   else
   {
     std::cerr << "database type unrecognized" << ENDL;
@@ -835,17 +808,7 @@ int main(int argc, char* argv[])
   }
 
   // for multi_db_compile:
-#else
-  if (db_engine_compiled != default_db_engine_compiled)
-  {
-    std::cerr << "Invalid database type for compiled version: " << db_type << std::endl;
-    return 1;
-  }
-#if BLOCKCHAIN_DB == DB_LMDB
   fake_core_db simple_core(m_config_folder, opt_testnet, opt_batch, db_type, db_flags);
-#else
-  fake_core_memory simple_core(m_config_folder, opt_testnet);
-#endif
 
   if (! vm["pop-blocks"].defaulted())
   {
@@ -856,17 +819,14 @@ int main(int argc, char* argv[])
     return 0;
   }
 
-#if !defined(BLOCKCHAIN_DB) || (BLOCKCHAIN_DB == DB_LMDB)
   if (! vm["drop-hard-fork"].defaulted())
   {
     LOG_PRINT_L0("Dropping hard fork tables...");
     simple_core.m_storage.get_db().drop_hard_fork_info();
     return 0;
   }
-#endif
 
   import_from_file(simple_core, import_file_path, block_stop);
-#endif
 
   }
   catch (const DB_ERROR& e)

src/blockchain_utilities/blockchain_utilities.h

@@ -31,21 +31,6 @@
 #include "version.h"
 
 
-// CONFIG: choose one of the three #define's
-//
-// DB_MEMORY was a sensible default for users migrating to LMDB, as it allowed
-// the exporter to use the in-memory blockchain while the other binaries
-// work with LMDB, without recompiling anything.
-// 
-// Now that the majority of users are on 0.9.2, and the converter has largely
-// become a generalised database tool, the default has changed to DB_LMDB.
-//
-// #define SOURCE_DB DB_MEMORY
-#define SOURCE_DB DB_LMDB
-// to use global compile-time setting (DB_MEMORY or DB_LMDB):
-// #define SOURCE_DB BLOCKCHAIN_DB
-
-
 // bounds checking is done before writing to buffer, but buffer size
 // should be a sensible maximum
 #define BUFFER_SIZE 1000000

src/blockchain_utilities/blocksdat_file.cpp

@@ -114,11 +114,7 @@ bool BlocksdatFile::close()
 }
 
 
-#if SOURCE_DB == DB_MEMORY
-bool BlocksdatFile::store_blockchain_raw(blockchain_storage* _blockchain_storage, tx_memory_pool* _tx_pool, boost::filesystem::path& output_file, uint64_t requested_block_stop)
-#else
 bool BlocksdatFile::store_blockchain_raw(Blockchain* _blockchain_storage, tx_memory_pool* _tx_pool, boost::filesystem::path& output_file, uint64_t requested_block_stop)
-#endif
 {
   uint64_t num_blocks_written = 0;
   m_blockchain_storage = _blockchain_storage;

src/blockchain_utilities/blocksdat_file.h

@@ -36,7 +36,6 @@
 
 #include "cryptonote_core/cryptonote_basic.h"
 #include "cryptonote_core/cryptonote_boost_serialization.h"
-#include "cryptonote_core/blockchain_storage.h"
 #include "cryptonote_core/blockchain.h"
 #include "blockchain_db/blockchain_db.h"
 #include "blockchain_db/lmdb/db_lmdb.h"
@@ -60,21 +59,12 @@ class BlocksdatFile
 {
 public:
 
-#if SOURCE_DB == DB_MEMORY
-  bool store_blockchain_raw(cryptonote::blockchain_storage* cs, cryptonote::tx_memory_pool* txp,
-      boost::filesystem::path& output_file, uint64_t use_block_height=0);
-#else
   bool store_blockchain_raw(cryptonote::Blockchain* cs, cryptonote::tx_memory_pool* txp,
       boost::filesystem::path& output_file, uint64_t use_block_height=0);
-#endif
 
 protected:
 
-#if SOURCE_DB == DB_MEMORY
-  blockchain_storage* m_blockchain_storage;
-#else
   Blockchain* m_blockchain_storage;
-#endif
 
   std::ofstream * m_raw_data_file;
 

src/blockchain_utilities/bootstrap_file.cpp

@@ -223,31 +223,9 @@ void BootstrapFile::write_block(block& block)
     {
       throw std::runtime_error("Aborting: tx == null_hash");
     }
-#if SOURCE_DB == DB_MEMORY
-    const transaction* tx = m_blockchain_storage->get_tx(tx_id);
-#else
     transaction tx = m_blockchain_storage->get_db().get_tx(tx_id);
-#endif
 
-#if SOURCE_DB == DB_MEMORY
-    if(tx == NULL)
-    {
-      if (! m_tx_pool)
-        throw std::runtime_error("Aborting: tx == NULL, so memory pool required to get tx, but memory pool isn't enabled");
-      else
-      {
-        transaction tx;
-        if(m_tx_pool->get_transaction(tx_id, tx))
     txs.push_back(tx);
-        else
-          throw std::runtime_error("Aborting: tx not found in pool");
-      }
-    }
-    else
-      txs.push_back(*tx);
-#else
-    txs.push_back(tx);
-#endif
   }
 
   // these non-coinbase txs will be serialized using this structure
@@ -257,15 +235,9 @@ void BootstrapFile::write_block(block& block)
   bool include_extra_block_data = true;
   if (include_extra_block_data)
   {
-#if SOURCE_DB == DB_MEMORY
-    size_t block_size = m_blockchain_storage->get_block_size(block_height);
-    difficulty_type cumulative_difficulty = m_blockchain_storage->get_block_cumulative_difficulty(block_height);
-    uint64_t coins_generated = m_blockchain_storage->get_block_coins_generated(block_height);
-#else
     size_t block_size = m_blockchain_storage->get_db().get_block_size(block_height);
     difficulty_type cumulative_difficulty = m_blockchain_storage->get_db().get_block_cumulative_difficulty(block_height);
     uint64_t coins_generated = m_blockchain_storage->get_db().get_block_already_generated_coins(block_height);
-#endif
 
     bp.block_size = block_size;
     bp.cumulative_difficulty = cumulative_difficulty;
@@ -288,11 +260,7 @@ bool BootstrapFile::close()
 }
 
 
-#if SOURCE_DB == DB_MEMORY
-bool BootstrapFile::store_blockchain_raw(blockchain_storage* _blockchain_storage, tx_memory_pool* _tx_pool, boost::filesystem::path& output_file, uint64_t requested_block_stop)
-#else
 bool BootstrapFile::store_blockchain_raw(Blockchain* _blockchain_storage, tx_memory_pool* _tx_pool, boost::filesystem::path& output_file, uint64_t requested_block_stop)
-#endif
 {
   uint64_t num_blocks_written = 0;
   m_max_chunk = 0;

src/blockchain_utilities/bootstrap_file.h

@@ -35,7 +35,6 @@
 #include <boost/iostreams/filtering_streambuf.hpp>
 
 #include "cryptonote_core/cryptonote_basic.h"
-#include "cryptonote_core/blockchain_storage.h"
 #include "cryptonote_core/blockchain.h"
 
 #include <algorithm>
@@ -60,21 +59,12 @@ public:
   uint64_t count_blocks(const std::string& dir_path);
   uint64_t seek_to_first_chunk(std::ifstream& import_file);
 
-#if SOURCE_DB == DB_MEMORY
-  bool store_blockchain_raw(cryptonote::blockchain_storage* cs, cryptonote::tx_memory_pool* txp,
-      boost::filesystem::path& output_file, uint64_t use_block_height=0);
-#else
   bool store_blockchain_raw(cryptonote::Blockchain* cs, cryptonote::tx_memory_pool* txp,
       boost::filesystem::path& output_file, uint64_t use_block_height=0);
-#endif
 
 protected:
 
-#if SOURCE_DB == DB_MEMORY
-  blockchain_storage* m_blockchain_storage;
-#else
   Blockchain* m_blockchain_storage;
-#endif
 
   tx_memory_pool* m_tx_pool;
   typedef std::vector<char> buffer_type;

src/blockchain_utilities/fake_core.h

@@ -30,7 +30,7 @@
 
 #include <boost/filesystem.hpp>
 #include "cryptonote_core/blockchain.h" // BlockchainDB
-#include "cryptonote_core/blockchain_storage.h" // in-memory DB
+#include "cryptonote_core/tx_pool.h"
 #include "blockchain_db/blockchain_db.h"
 #include "blockchain_db/lmdb/db_lmdb.h"
 #if defined(BERKELEY_DB)
@@ -48,8 +48,6 @@ namespace
 }
 
 
-#if !defined(BLOCKCHAIN_DB) || BLOCKCHAIN_DB == DB_LMDB
-
 struct fake_core_db
 {
   Blockchain m_storage;
@@ -59,12 +57,7 @@ struct fake_core_db
   bool support_add_block;
 
   // for multi_db_runtime:
-#if !defined(BLOCKCHAIN_DB)
-  fake_core_db(const boost::filesystem::path &path, const bool use_testnet=false, const bool do_batch=true, const std::string& db_type="lmdb", const int db_flags=0) : m_pool(&m_storage), m_storage(m_pool)
-  // for multi_db_compile:
-#else
   fake_core_db(const boost::filesystem::path &path, const bool use_testnet=false, const bool do_batch=true, const std::string& db_type="lmdb", const int db_flags=0) : m_pool(m_storage), m_storage(m_pool)
-#endif
   {
     m_pool.init(path.string());
 
@@ -137,59 +130,4 @@ struct fake_core_db
   }
 
 };
-#endif
 
-#if !defined(BLOCKCHAIN_DB) || BLOCKCHAIN_DB == DB_MEMORY
-
-struct fake_core_memory
-{
-  blockchain_storage m_storage;
-  tx_memory_pool m_pool;
-  bool support_batch;
-  bool support_add_block;
-
-  // for multi_db_runtime:
-#if !defined(BLOCKCHAIN_DB)
-  fake_core_memory(const boost::filesystem::path &path, const bool use_testnet=false) : m_pool(&m_storage), m_storage(m_pool)
-#else
-  // for multi_db_compile:
-  fake_core_memory(const boost::filesystem::path &path, const bool use_testnet=false) : m_pool(m_storage), m_storage(&m_pool)
-#endif
-  {
-    m_pool.init(path.string());
-    m_storage.init(path.string(), use_testnet);
-    support_batch = false;
-    support_add_block = false;
-  }
-  ~fake_core_memory()
-  {
-    LOG_PRINT_L3("fake_core_memory() destructor called - want to see it ripple down");
-    m_storage.deinit();
-  }
-
-  uint64_t add_block(const block& blk
-                            , const size_t& block_size
-                            , const difficulty_type& cumulative_difficulty
-                            , const uint64_t& coins_generated
-                            , const std::vector<transaction>& txs
-                            )
-  {
-    // TODO:
-    // would need to refactor handle_block_to_main_chain() to have a direct add_block() method like Blockchain class
-    throw std::runtime_error("direct add_block() method not implemented for in-memory db");
-    return 2;
-  }
-
-  void batch_start(uint64_t batch_num_blocks = 0)
-  {
-    LOG_PRINT_L0("WARNING: [batch_start] opt_batch set, but this database doesn't support/need transactions - ignoring");
-  }
-
-  void batch_stop()
-  {
-    LOG_PRINT_L0("WARNING: [batch_stop] opt_batch set, but this database doesn't support/need transactions - ignoring");
-  }
-
-};
-
-#endif

src/common/json_util.h

@@ -28,8 +28,8 @@
 
 #pragma once 
 
-#define GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, name, type, jtype, mandatory) \
+#define GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, name, type, jtype, mandatory, def) \
-  type field_##name; \
+  type field_##name = def; \
   bool field_##name##_found = false; \
   (void)field_##name##_found; \
   do if (json.HasMember(#name)) \

src/crypto/crypto-ops.c

@@ -40,17 +40,15 @@ DISABLE_VS_WARNINGS(4146 4244)
 
 static void fe_mul(fe, const fe, const fe);
 static void fe_sq(fe, const fe);
-static void fe_tobytes(unsigned char *, const fe);
 static void ge_madd(ge_p1p1 *, const ge_p3 *, const ge_precomp *);
 static void ge_msub(ge_p1p1 *, const ge_p3 *, const ge_precomp *);
 static void ge_p2_0(ge_p2 *);
 static void ge_p3_dbl(ge_p1p1 *, const ge_p3 *);
-static void ge_sub(ge_p1p1 *, const ge_p3 *, const ge_cached *);
 static void fe_divpowm1(fe, const fe, const fe);
 
 /* Common functions */
 
-static uint64_t load_3(const unsigned char *in) {
+uint64_t load_3(const unsigned char *in) {
   uint64_t result;
   result = (uint64_t) in[0];
   result |= ((uint64_t) in[1]) << 8;
@@ -58,7 +56,7 @@ static uint64_t load_3(const unsigned char *in) {
   return result;
 }
 
-static uint64_t load_4(const unsigned char *in)
+uint64_t load_4(const unsigned char *in)
 {
   uint64_t result;
   result = (uint64_t) in[0];
@@ -120,7 +118,7 @@ Postconditions:
    |h| bounded by 1.1*2^26,1.1*2^25,1.1*2^26,1.1*2^25,etc.
 */
 
-static void fe_add(fe h, const fe f, const fe g) {
+void fe_add(fe h, const fe f, const fe g) {
   int32_t f0 = f[0];
   int32_t f1 = f[1];
   int32_t f2 = f[2];
@@ -258,7 +256,7 @@ static void fe_copy(fe h, const fe f) {
 
 /* From fe_invert.c */
 
-static void fe_invert(fe out, const fe z) {
+void fe_invert(fe out, const fe z) {
   fe t0;
   fe t1;
   fe t2;
@@ -1031,7 +1029,7 @@ Proof:
   so floor(2^(-255)(h + 19 2^(-25) h9 + 2^(-1))) = q.
 */
 
-static void fe_tobytes(unsigned char *s, const fe h) {
+void fe_tobytes(unsigned char *s, const fe h) {
   int32_t h0 = h[0];
   int32_t h1 = h[1];
   int32_t h2 = h[2];
@@ -1591,7 +1589,7 @@ void ge_scalarmult_base(ge_p3 *h, const unsigned char *a) {
 r = p - q
 */
 
-static void ge_sub(ge_p1p1 *r, const ge_p3 *p, const ge_cached *q) {
+void ge_sub(ge_p1p1 *r, const ge_p3 *p, const ge_cached *q) {
   fe t0;
   fe_add(r->X, p->Y, p->X);
   fe_sub(r->Y, p->Y, p->X);

src/crypto/crypto-ops.h

@@ -143,3 +143,11 @@ void sc_sub(unsigned char *, const unsigned char *, const unsigned char *);
 void sc_mulsub(unsigned char *, const unsigned char *, const unsigned char *, const unsigned char *);
 int sc_check(const unsigned char *);
 int sc_isnonzero(const unsigned char *); /* Doesn't normalize */
+
+// internal
+uint64_t load_3(const unsigned char *in);
+uint64_t load_4(const unsigned char *in);
+void ge_sub(ge_p1p1 *r, const ge_p3 *p, const ge_cached *q);
+void fe_add(fe h, const fe f, const fe g);
+void fe_tobytes(unsigned char *, const fe);
+void fe_invert(fe out, const fe z);

src/crypto/crypto.cpp

@@ -151,7 +151,7 @@ namespace crypto {
     return true;
   }
 
-  static void derivation_to_scalar(const key_derivation &derivation, size_t output_index, ec_scalar &res) {
+  void crypto_ops::derivation_to_scalar(const key_derivation &derivation, size_t output_index, ec_scalar &res) {
     struct {
       key_derivation derivation;
       char output_index[(sizeof(size_t) * 8 + 6) / 7];
@@ -230,7 +230,7 @@ namespace crypto {
     buf.h = prefix_hash;
     buf.key = pub;
     if (ge_frombytes_vartime(&tmp3, &pub) != 0) {
-      abort();
+      return false;
     }
     if (sc_check(&sig.c) != 0 || sc_check(&sig.r) != 0) {
       return false;
@@ -364,7 +364,7 @@ POP_WARNINGS
         return false;
       }
       if (ge_frombytes_vartime(&tmp3, &*pubs[i]) != 0) {
-        abort();
+        return false;
       }
       ge_double_scalarmult_base_vartime(&tmp2, &sig[i].c, &tmp3, &sig[i].r);
       ge_tobytes(&buf->ab[i].a, &tmp2);

src/crypto/crypto.h

@@ -64,6 +64,22 @@ namespace crypto {
     friend class crypto_ops;
   };
 
+  POD_CLASS public_keyV {
+    std::vector<public_key> keys;
+    int rows;
+  };
+
+  POD_CLASS secret_keyV {
+    std::vector<secret_key> keys;
+    int rows;
+  };
+
+  POD_CLASS public_keyM {
+    int cols;
+    int rows;
+    std::vector<secret_keyV> column_vectors;
+  };
+
   POD_CLASS key_derivation: ec_point {
     friend class crypto_ops;
   };
@@ -97,6 +113,8 @@ namespace crypto {
     friend bool secret_key_to_public_key(const secret_key &, public_key &);
     static bool generate_key_derivation(const public_key &, const secret_key &, key_derivation &);
     friend bool generate_key_derivation(const public_key &, const secret_key &, key_derivation &);
+    static void derivation_to_scalar(const key_derivation &derivation, size_t output_index, ec_scalar &res);
+    friend void derivation_to_scalar(const key_derivation &derivation, size_t output_index, ec_scalar &res);
     static bool derive_public_key(const key_derivation &, std::size_t, const public_key &, public_key &);
     friend bool derive_public_key(const key_derivation &, std::size_t, const public_key &, public_key &);
     static void derive_secret_key(const key_derivation &, std::size_t, const secret_key &, secret_key &);
@@ -165,6 +183,9 @@ namespace crypto {
     const public_key &base, public_key &derived_key) {
     return crypto_ops::derive_public_key(derivation, output_index, base, derived_key);
   }
+  inline void derivation_to_scalar(const key_derivation &derivation, size_t output_index, ec_scalar &res) {
+    return crypto_ops::derivation_to_scalar(derivation, output_index, res);
+  }
   inline void derive_secret_key(const key_derivation &derivation, std::size_t output_index,
     const secret_key &base, secret_key &derived_key) {
     crypto_ops::derive_secret_key(derivation, output_index, base, derived_key);

src/crypto/keccak.c

@@ -73,11 +73,11 @@ void keccakf(uint64_t st[25], int rounds)
 // compute a keccak hash (md) of given byte length from "in"
 typedef uint64_t state_t[25];
 
-int keccak(const uint8_t *in, int inlen, uint8_t *md, int mdlen)
+int keccak(const uint8_t *in, size_t inlen, uint8_t *md, int mdlen)
 {
     state_t st;
     uint8_t temp[144];
-    int i, rsiz, rsizw;
+    size_t i, rsiz, rsizw;
 
     rsiz = sizeof(state_t) == mdlen ? HASH_DATA_AREA : 200 - 2 * mdlen;
     rsizw = rsiz / 8;
@@ -106,7 +106,7 @@ int keccak(const uint8_t *in, int inlen, uint8_t *md, int mdlen)
     return 0;
 }
 
-void keccak1600(const uint8_t *in, int inlen, uint8_t *md)
+void keccak1600(const uint8_t *in, size_t inlen, uint8_t *md)
 {
     keccak(in, inlen, md, sizeof(state_t));
 }

src/crypto/keccak.h

@@ -16,11 +16,11 @@
 #endif
 
 // compute a keccak hash (md) of given byte length from "in"
-int keccak(const uint8_t *in, int inlen, uint8_t *md, int mdlen);
+int keccak(const uint8_t *in, size_t inlen, uint8_t *md, int mdlen);
 
 // update the state
 void keccakf(uint64_t st[25], int norounds);
 
-void keccak1600(const uint8_t *in, int inlen, uint8_t *md);
+void keccak1600(const uint8_t *in, size_t inlen, uint8_t *md);
 
 #endif

src/cryptonote_config.h

@@ -41,7 +41,7 @@
 #define CRYPTONOTE_MAX_TX_SIZE                          1000000000
 #define CRYPTONOTE_PUBLIC_ADDRESS_TEXTBLOB_VER          0
 #define CRYPTONOTE_MINED_MONEY_UNLOCK_WINDOW            60
-#define CURRENT_TRANSACTION_VERSION                     1
+#define CURRENT_TRANSACTION_VERSION                     2
 #define CURRENT_BLOCK_MAJOR_VERSION                     1
 #define CURRENT_BLOCK_MINOR_VERSION                     0
 #define CRYPTONOTE_BLOCK_FUTURE_TIME_LIMIT              60*60*2

src/cryptonote_core/CMakeLists.txt

@@ -28,7 +28,6 @@
 
 set(cryptonote_core_sources
   account.cpp
-  blockchain_storage.cpp
   blockchain.cpp
   checkpoints.cpp
   cryptonote_basic_impl.cpp
@@ -44,7 +43,6 @@ set(cryptonote_core_headers)
 set(cryptonote_core_private_headers
   account.h
   account_boost_serialization.h
-  blockchain_storage.h
   blockchain_storage_boost_serialization.h
   blockchain.h
   checkpoints.h
@@ -80,6 +78,7 @@ target_link_libraries(cryptonote_core
     crypto
     otshell_utils
     blockchain_db
+    ringct
     ${Boost_DATE_TIME_LIBRARY}
     ${Boost_PROGRAM_OPTIONS_LIBRARY}
     ${Boost_SERIALIZATION_LIBRARY}

src/cryptonote_core/blockchain.cpp

@@ -51,6 +51,7 @@
 #include "crypto/hash.h"
 #include "cryptonote_core/checkpoints.h"
 #include "cryptonote_core/cryptonote_core.h"
+#include "ringct/rctSigs.h"
 #if defined(PER_BLOCK_CHECKPOINT)
 #include "blocks/blocks.h"
 #endif
@@ -98,6 +99,10 @@ static const struct {
 
   // version 2 starts from block 624634, which is on or around the 23rd of November, 2015. Fork time finalised on 2015-11-20. No fork voting occurs for the v2 fork.
   { 2, 624634, 0, 1445355000 },
+
+  { 3, 800500, 0, 1472415034 },
+  { 4, 801220, 0, 1472415035 },
+  { 5, 802660, 0, 1472415036 },
 };
 static const uint64_t testnet_hard_fork_version_1_till = 624633;
 
@@ -127,7 +132,7 @@ bool Blockchain::have_tx_keyimg_as_spent(const crypto::key_image &key_im) const
 // and collects the public key for each from the transaction it was included in
 // via the visitor passed to it.
 template <class visitor_t>
-bool Blockchain::scan_outputkeys_for_indexes(const txin_to_key& tx_in_to_key, visitor_t &vis, const crypto::hash &tx_prefix_hash, uint64_t* pmax_related_block_height) const
+bool Blockchain::scan_outputkeys_for_indexes(size_t tx_version, const txin_to_key& tx_in_to_key, visitor_t &vis, const crypto::hash &tx_prefix_hash, uint64_t* pmax_related_block_height) const
 {
   LOG_PRINT_L3("Blockchain::" << __func__);
 
@@ -207,7 +212,7 @@ bool Blockchain::scan_outputkeys_for_indexes(const txin_to_key& tx_in_to_key, vi
           output_index = m_db->get_output_key(tx_in_to_key.amount, i);
 
         // call to the passed boost visitor to grab the public key for the output
-        if (!vis.handle_output(output_index.unlock_time, output_index.pubkey))
+        if (!vis.handle_output(output_index.unlock_time, output_index.pubkey, output_index.commitment))
         {
           LOG_PRINT_L0("Failed to handle_output for output no = " << count << ", with absolute offset " << i);
           return false;
@@ -958,7 +963,7 @@ bool Blockchain::validate_miner_transaction(const block& b, size_t cumulative_bl
     money_in_use += o.amount;
   partial_block_reward = false;
 
-  if (version >= 3) {
+  if (version == 3) {
     for (auto &o: b.miner_tx.vout) {
       if (!is_valid_decomposed_amount(o.amount)) {
         LOG_PRINT_L1("miner tx output " << print_money(o.amount) << " is not a valid decomposed amount");
@@ -969,7 +974,7 @@ bool Blockchain::validate_miner_transaction(const block& b, size_t cumulative_bl
 
   std::vector<size_t> last_blocks_sizes;
   get_last_n_blocks_sizes(last_blocks_sizes, CRYPTONOTE_REWARD_BLOCKS_WINDOW);
-  if (!get_block_reward(epee::misc_utils::median(last_blocks_sizes), cumulative_block_size, already_generated_coins, base_reward, get_current_hard_fork_version()))
+  if (!get_block_reward(epee::misc_utils::median(last_blocks_sizes), cumulative_block_size, already_generated_coins, base_reward, version))
   {
     LOG_PRINT_L1("block size " << cumulative_block_size << " is bigger than allowed for this blockchain");
     return false;
@@ -1086,6 +1091,8 @@ bool Blockchain::create_block_template(block& b, const account_public_address& m
     {
       LOG_ERROR("Creating block template: error: invalid transaction size");
     }
+    if (cur_tx.tx.version == 1)
+    {
       uint64_t inputs_amount;
       if (!get_inputs_money_amount(cur_tx.tx, inputs_amount))
       {
@@ -1096,6 +1103,14 @@ bool Blockchain::create_block_template(block& b, const account_public_address& m
         LOG_ERROR("Creating block template: error: invalid fee");
       }
     }
+    else
+    {
+      if (cur_tx.fee != cur_tx.tx.txnFee)
+      {
+        LOG_ERROR("Creating block template: error: invalid fee");
+      }
+    }
+  }
   if (txs_size != real_txs_size)
   {
     LOG_ERROR("Creating block template: error: wrongly calculated transaction size");
@@ -1117,7 +1132,9 @@ bool Blockchain::create_block_template(block& b, const account_public_address& m
    block size, so first miner transaction generated with fake amount of money, and with phase we know think we know expected block size
    */
   //make blocks coin-base tx looks close to real coinbase tx to get truthful blob size
-  bool r = construct_miner_tx(height, median_size, already_generated_coins, txs_size, fee, miner_address, b.miner_tx, ex_nonce, 11, m_hardfork->get_current_version());
+  uint8_t hf_version = m_hardfork->get_current_version();
+  size_t max_outs = hf_version >= 4 ? 1 : 11;
+  bool r = construct_miner_tx(height, median_size, already_generated_coins, txs_size, fee, miner_address, b.miner_tx, ex_nonce, max_outs, hf_version);
   CHECK_AND_ASSERT_MES(r, false, "Failed to construc miner tx, first chance");
   size_t cumulative_size = txs_size + get_object_blobsize(b.miner_tx);
 #if defined(DEBUG_CREATE_BLOCK_TEMPLATE)
@@ -1126,7 +1143,7 @@ bool Blockchain::create_block_template(block& b, const account_public_address& m
 #endif
   for (size_t try_count = 0; try_count != 10; ++try_count)
   {
-    r = construct_miner_tx(height, median_size, already_generated_coins, cumulative_size, fee, miner_address, b.miner_tx, ex_nonce, 11, m_hardfork->get_current_version());
+    r = construct_miner_tx(height, median_size, already_generated_coins, cumulative_size, fee, miner_address, b.miner_tx, ex_nonce, max_outs, hf_version);
 
     CHECK_AND_ASSERT_MES(r, false, "Failed to construc miner tx, second chance");
     size_t coinbase_blob_size = get_object_blobsize(b.miner_tx);
@@ -1597,6 +1614,139 @@ bool Blockchain::get_random_outs_for_amounts(const COMMAND_RPC_GET_RANDOM_OUTPUT
   return true;
 }
 //------------------------------------------------------------------
+// This function adds the ringct output at index i to the list
+// unlocked and other such checks should be done by here.
+void Blockchain::add_out_to_get_rct_random_outs(std::list<COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::out_entry>& outs, uint64_t amount, size_t i) const
+{
+  LOG_PRINT_L3("Blockchain::" << __func__);
+  CRITICAL_REGION_LOCAL(m_blockchain_lock);
+
+  COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::out_entry& oen = *outs.insert(outs.end(), COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::out_entry());
+  oen.amount = amount;
+  oen.global_amount_index = i;
+  output_data_t data = m_db->get_output_key(amount, i);
+  oen.out_key = data.pubkey;
+  oen.commitment = data.commitment;
+}
+//------------------------------------------------------------------
+// This function takes an RPC request for mixins and creates an RPC response
+// with the requested mixins.
+// TODO: figure out why this returns boolean / if we should be returning false
+// in some cases
+bool Blockchain::get_random_rct_outs(const COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::request& req, COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::response& res) const
+{
+  LOG_PRINT_L3("Blockchain::" << __func__);
+  CRITICAL_REGION_LOCAL(m_blockchain_lock);
+
+  // for each amount that we need to get mixins for, get <n> random outputs
+  // from BlockchainDB where <n> is req.outs_count (number of mixins).
+  auto num_outs = m_db->get_num_outputs(0);
+  // ensure we don't include outputs that aren't yet eligible to be used
+  // outpouts are sorted by height
+  while (num_outs > 0)
+  {
+    const tx_out_index toi = m_db->get_output_tx_and_index(0, num_outs - 1);
+    const uint64_t height = m_db->get_tx_block_height(toi.first);
+    if (height + CRYPTONOTE_DEFAULT_TX_SPENDABLE_AGE <= m_db->height())
+      break;
+    --num_outs;
+  }
+
+  std::unordered_set<uint64_t> seen_indices;
+
+  // if there aren't enough outputs to mix with (or just enough),
+  // use all of them.  Eventually this should become impossible.
+  if (num_outs <= req.outs_count)
+  {
+    for (uint64_t i = 0; i < num_outs; i++)
+    {
+      // get tx_hash, tx_out_index from DB
+      tx_out_index toi = m_db->get_output_tx_and_index(0, i);
+
+      // if tx is unlocked, add output to result_outs
+      if (is_tx_spendtime_unlocked(m_db->get_tx_unlock_time(toi.first)))
+      {
+        add_out_to_get_rct_random_outs(res.outs, 0, i);
+      }
+    }
+  }
+  else
+  {
+    // while we still need more mixins
+    while (res.outs.size() < req.outs_count)
+    {
+      // if we've gone through every possible output, we've gotten all we can
+      if (seen_indices.size() == num_outs)
+      {
+        break;
+      }
+
+      // get a random output index from the DB.  If we've already seen it,
+      // return to the top of the loop and try again, otherwise add it to the
+      // list of output indices we've seen.
+
+      // triangular distribution over [a,b) with a=0, mode c=b=up_index_limit
+      uint64_t r = crypto::rand<uint64_t>() % ((uint64_t)1 << 53);
+      double frac = std::sqrt((double)r / ((uint64_t)1 << 53));
+      uint64_t i = (uint64_t)(frac*num_outs);
+      // just in case rounding up to 1 occurs after sqrt
+      if (i == num_outs)
+        --i;
+
+      if (seen_indices.count(i))
+      {
+        continue;
+      }
+      seen_indices.emplace(i);
+
+      // get tx_hash, tx_out_index from DB
+      tx_out_index toi = m_db->get_output_tx_and_index(0, i);
+
+      // if the output's transaction is unlocked, add the output's index to
+      // our list.
+      if (is_tx_spendtime_unlocked(m_db->get_tx_unlock_time(toi.first)))
+      {
+        add_out_to_get_rct_random_outs(res.outs, 0, i);
+      }
+    }
+  }
+
+  if (res.outs.size() < req.outs_count)
+    return false;
+#if 0
+  // if we do not have enough RCT inputs, we can pick from the non RCT ones
+  // which will have a zero mask
+  if (res.outs.size() < req.outs_count)
+  {
+    LOG_PRINT_L0("Out of RCT inputs (" << res.outs.size() << "/" << req.outs_count << "), using regular ones");
+
+    // TODO: arbitrary selection, needs better
+    COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::request req2 = AUTO_VAL_INIT(req2);
+    COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::response res2 = AUTO_VAL_INIT(res2);
+    req2.outs_count = req.outs_count - res.outs.size();
+    static const uint64_t amounts[] = {1, 10, 20, 50, 100, 200, 500, 1000, 10000};
+    for (uint64_t a: amounts)
+      req2.amounts.push_back(a);
+    if (!get_random_outs_for_amounts(req2, res2))
+      return false;
+
+    // pick random ones from there
+    while (res.outs.size() < req.outs_count)
+    {
+      int list_idx = rand() % (sizeof(amounts)/sizeof(amounts[0]));
+      if (!res2.outs[list_idx].outs.empty())
+      {
+        const COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::out_entry oe = res2.outs[list_idx].outs.back();
+        res2.outs[list_idx].outs.pop_back();
+        add_out_to_get_rct_random_outs(res.outs, res2.outs[list_idx].amount, oe.global_amount_index);
+      }
+    }
+  }
+#endif
+
+  return true;
+}
+//------------------------------------------------------------------
 bool Blockchain::get_outs(const COMMAND_RPC_GET_OUTPUTS::request& req, COMMAND_RPC_GET_OUTPUTS::response& res) const
 {
   LOG_PRINT_L3("Blockchain::" << __func__);
@@ -1607,11 +1757,11 @@ bool Blockchain::get_outs(const COMMAND_RPC_GET_OUTPUTS::request& req, COMMAND_R
   for (const auto &i: req.outputs)
   {
     // get tx_hash, tx_out_index from DB
-    crypto::public_key key = m_db->get_output_key(i.amount, i.index).pubkey;
+    const output_data_t &od = m_db->get_output_key(i.amount, i.index);
     tx_out_index toi = m_db->get_output_tx_and_index(i.amount, i.index);
     bool unlocked = is_tx_spendtime_unlocked(m_db->get_tx_unlock_time(toi.first));
 
-    res.outs.push_back({key, unlocked});
+    res.outs.push_back({od.pubkey, od.commitment, unlocked});
   }
   return true;
 }
@@ -2012,7 +2162,7 @@ bool Blockchain::get_tx_outputs_gindexs(const crypto::hash& tx_id, std::vector<u
 // This function overloads its sister function with
 // an extra value (hash of highest block that holds an output used as input)
 // as a return-by-reference.
-bool Blockchain::check_tx_inputs(const transaction& tx, uint64_t& max_used_block_height, crypto::hash& max_used_block_id, tx_verification_context &tvc, bool kept_by_block)
+bool Blockchain::check_tx_inputs(transaction& tx, uint64_t& max_used_block_height, crypto::hash& max_used_block_id, tx_verification_context &tvc, bool kept_by_block)
 {
   LOG_PRINT_L3("Blockchain::" << __func__);
   CRITICAL_REGION_LOCAL(m_blockchain_lock);
@@ -2055,12 +2205,27 @@ bool Blockchain::check_tx_outputs(const transaction& tx, tx_verification_context
   // from hard fork 2, we forbid dust and compound outputs
   if (m_hardfork->get_current_version() >= 2) {
     for (auto &o: tx.vout) {
+      if (tx.version == 1)
+      {
         if (!is_valid_decomposed_amount(o.amount)) {
           tvc.m_invalid_output = true;
           return false;
         }
       }
     }
+  }
+
+  // in a v2 tx, all outputs must have 0 amount
+  if (m_hardfork->get_current_version() >= 3) {
+    if (tx.version >= 2) {
+      for (auto &o: tx.vout) {
+        if (o.amount != 0) {
+          tvc.m_invalid_output = true;
+          return false;
+        }
+      }
+    }
+  }
 
   return true;
 }
@@ -2076,13 +2241,83 @@ bool Blockchain::have_tx_keyimges_as_spent(const transaction &tx) const
   }
   return false;
 }
+bool Blockchain::expand_transaction_2(transaction &tx, const crypto::hash &tx_prefix_hash, const std::vector<std::vector<rct::ctkey>> &pubkeys)
+{
+  CHECK_AND_ASSERT_MES(tx.version == 2, false, "Transaction version is not 2");
+
+  rct::rctSig &rv = tx.rct_signatures;
+
+  // message - hash of the transaction prefix
+  rv.message = rct::hash2rct(tx_prefix_hash);
+
+  // mixRing - full and simple store it in opposite ways
+  if (rv.type == rct::RCTTypeFull)
+  {
+    rv.mixRing.resize(pubkeys[0].size());
+    for (size_t m = 0; m < pubkeys[0].size(); ++m)
+      rv.mixRing[m].clear();
+    for (size_t n = 0; n < pubkeys.size(); ++n)
+    {
+      CHECK_AND_ASSERT_MES(pubkeys[n].size() <= pubkeys[0].size(), false, "More inputs that first ring");
+      for (size_t m = 0; m < pubkeys[n].size(); ++m)
+      {
+        rv.mixRing[m].push_back(pubkeys[n][m]);
+      }
+    }
+  }
+  else if (rv.type == rct::RCTTypeSimple)
+  {
+    rv.mixRing.resize(pubkeys.size());
+    for (size_t n = 0; n < pubkeys.size(); ++n)
+    {
+      rv.mixRing[n].clear();
+      for (size_t m = 0; m < pubkeys[n].size(); ++m)
+      {
+        rv.mixRing[n].push_back(pubkeys[n][m]);
+      }
+    }
+  }
+  else
+  {
+    CHECK_AND_ASSERT_MES(false, false, "Unsupported rct tx type: " + boost::lexical_cast<std::string>(rv.type));
+  }
+
+  // II
+  if (rv.type == rct::RCTTypeFull)
+  {
+    rv.p.MGs.resize(1);
+    rv.p.MGs[0].II.resize(tx.vin.size());
+    for (size_t n = 0; n < tx.vin.size(); ++n)
+      rv.p.MGs[0].II[n] = rct::ki2rct(boost::get<txin_to_key>(tx.vin[n]).k_image);
+  }
+  else if (rv.type == rct::RCTTypeSimple)
+  {
+    CHECK_AND_ASSERT_MES(rv.p.MGs.size() == tx.vin.size(), false, "Bad MGs size");
+    for (size_t n = 0; n < tx.vin.size(); ++n)
+    {
+      rv.p.MGs[n].II.resize(1);
+      rv.p.MGs[n].II[0] = rct::ki2rct(boost::get<txin_to_key>(tx.vin[n]).k_image);
+    }
+  }
+  else
+  {
+    CHECK_AND_ASSERT_MES(false, false, "Unsupported rct tx type: " + boost::lexical_cast<std::string>(rv.type));
+  }
+
+  // outPk
+  CHECK_AND_ASSERT_MES(rv.outPk.size() == tx.vout.size(), false, "Bad outPk size");
+  for (size_t n = 0; n < tx.rct_signatures.outPk.size(); ++n)
+    rv.outPk[n].dest = rct::pk2rct(boost::get<txout_to_key>(tx.vout[n].target).key);
+
+  return true;
+}
 //------------------------------------------------------------------
 // This function validates transaction inputs and their keys.
 // FIXME: consider moving functionality specific to one input into
 //        check_tx_input() rather than here, and use this function simply
 //        to iterate the inputs as necessary (splitting the task
 //        using threads, etc.)
-bool Blockchain::check_tx_inputs(const transaction& tx, tx_verification_context &tvc, uint64_t* pmax_used_block_height)
+bool Blockchain::check_tx_inputs(transaction& tx, tx_verification_context &tvc, uint64_t* pmax_used_block_height)
 {
   LOG_PRINT_L3("Blockchain::" << __func__);
   size_t sig_index = 0;
@@ -2091,30 +2326,44 @@ bool Blockchain::check_tx_inputs(const transaction& tx, tx_verification_context
 
   crypto::hash tx_prefix_hash = get_transaction_prefix_hash(tx);
 
+  const uint8_t hf_version = m_hardfork->get_current_version();
+
   // from hard fork 2, we require mixin at least 2 unless one output cannot mix with 2 others
   // if one output cannot mix with 2 others, we accept at most 1 output that can mix
-  if (m_hardfork->get_current_version() >= 2)
+  if (hf_version >= 2)
   {
     size_t n_unmixable = 0, n_mixable = 0;
     size_t mixin = std::numeric_limits<size_t>::max();
+    const size_t min_mixin = hf_version >= 5 ? 4 : 2;
     for (const auto& txin : tx.vin)
     {
       // non txin_to_key inputs will be rejected below
       if (txin.type() == typeid(txin_to_key))
       {
         const txin_to_key& in_to_key = boost::get<txin_to_key>(txin);
+        if (in_to_key.amount == 0)
+        {
+          // always consider rct inputs mixable. Even if there's not enough rct
+          // inputs on the chain to mix with, this is going to be the case for
+          // just a few blocks right after the fork at most
+          ++n_mixable;
+        }
+        else
+        {
           uint64_t n_outputs = m_db->get_num_outputs(in_to_key.amount);
           LOG_PRINT_L2("output size " << print_money(in_to_key.amount) << ": " << n_outputs << " available");
           // n_outputs includes the output we're considering
-        if (n_outputs <= 2)
+          if (n_outputs <= min_mixin)
             ++n_unmixable;
           else
             ++n_mixable;
+        }
         if (in_to_key.key_offsets.size() - 1 < mixin)
           mixin = in_to_key.key_offsets.size() - 1;
       }
     }
-    if (mixin < 2)
+
+    if (mixin < min_mixin)
     {
       if (n_unmixable == 0)
       {
@@ -2129,6 +2378,22 @@ bool Blockchain::check_tx_inputs(const transaction& tx, tx_verification_context
         return false;
       }
     }
+
+    // min/max tx version based on HF, and we accept v1 txes if having a non mixable
+    const size_t max_tx_version = (hf_version <= 3) ? 1 : 2;
+    if (tx.version > max_tx_version)
+    {
+      LOG_PRINT_L1("transaction version " << (unsigned)tx.version << " is higher than max accepted version " << max_tx_version);
+      tvc.m_verifivation_failed = true;
+      return false;
+    }
+    const size_t min_tx_version = (n_unmixable > 0 ? 1 : (hf_version >= 5) ? 2 : 1);
+    if (tx.version < min_tx_version)
+    {
+      LOG_PRINT_L1("transaction version " << (unsigned)tx.version << " is lower than min accepted version " << min_tx_version);
+      tvc.m_verifivation_failed = true;
+      return false;
+    }
   }
 
   auto it = m_check_txin_table.find(tx_prefix_hash);
@@ -2140,7 +2405,7 @@ bool Blockchain::check_tx_inputs(const transaction& tx, tx_verification_context
   }
 
   uint64_t t_t1 = 0;
-  std::vector<std::vector<crypto::public_key>> pubkeys(tx.vin.size());
+  std::vector<std::vector<rct::ctkey>> pubkeys(tx.vin.size());
   std::vector < uint64_t > results;
   results.resize(tx.vin.size(), 0);
 
@@ -2189,6 +2454,8 @@ bool Blockchain::check_tx_inputs(const transaction& tx, tx_verification_context
       return false;
     }
 
+    if (tx.version == 1)
+    {
       // basically, make sure number of inputs == number of signatures
       CHECK_AND_ASSERT_MES(sig_index < tx.signatures.size(), false, "wrong transaction: not signature entry for input with index= " << sig_index);
 
@@ -2207,10 +2474,11 @@ bool Blockchain::check_tx_inputs(const transaction& tx, tx_verification_context
         continue;
       }
 #endif
+    }
 
     // make sure that output being spent matches up correctly with the
     // signature spending it.
-    if (!check_tx_input(in_to_key, tx_prefix_hash, tx.signatures[sig_index], pubkeys[sig_index], pmax_used_block_height))
+    if (!check_tx_input(tx.version, in_to_key, tx_prefix_hash, tx.version == 1 ? tx.signatures[sig_index] : std::vector<crypto::signature>(), tx.rct_signatures, pubkeys[sig_index], pmax_used_block_height))
     {
       it->second[in_to_key.k_image] = false;
       LOG_PRINT_L1("Failed to check ring signature for tx " << get_transaction_hash(tx) << "  vin key with k_image: " << in_to_key.k_image << "  sig_index: " << sig_index);
@@ -2222,6 +2490,8 @@ bool Blockchain::check_tx_inputs(const transaction& tx, tx_verification_context
       return false;
     }
 
+    if (tx.version == 1)
+    {
       if (threads > 1)
       {
         // ND: Speedup
@@ -2245,12 +2515,15 @@ bool Blockchain::check_tx_inputs(const transaction& tx, tx_verification_context
         }
         it->second[in_to_key.k_image] = true;
       }
+    }
 
     sig_index++;
   }
 
   KILL_IOSERVICE();
 
+  if (tx.version == 1)
+  {
     if (threads > 1)
     {
       // save results to table, passed or otherwise
@@ -2269,12 +2542,150 @@ bool Blockchain::check_tx_inputs(const transaction& tx, tx_verification_context
         return false;
       }
     }
-  LOG_PRINT_L1("t_loop: " << t_t1);
+  }
+  else
+  {
+    if (!expand_transaction_2(tx, tx_prefix_hash, pubkeys))
+    {
+      LOG_PRINT_L1("Failed to expand rct signatures!");
+      return false;
+    }
+
+    // from version 2, check ringct signatures
+    // obviously, the original and simple rct APIs use a mixRing that's indexes
+    // in opposite orders, because it'd be too simple otherwise...
+    const rct::rctSig &rv = tx.rct_signatures;
+    switch (rv.type)
+    {
+    case rct::RCTTypeNull: {
+      // we only accept no signatures for coinbase txes
+      LOG_PRINT_L1("Null rct signature on non-coinbase tx");
+      return false;
+    }
+    case rct::RCTTypeSimple: {
+      // check all this, either recontructed (so should really pass), or not
+      {
+        if (pubkeys.size() != rv.mixRing.size())
+        {
+          LOG_PRINT_L1("Failed to check ringct signatures: mismatched pubkeys/mixRing size");
+          return false;
+        }
+        for (size_t i = 0; i < pubkeys.size(); ++i)
+        {
+          if (pubkeys[i].size() != rv.mixRing[i].size())
+          {
+            LOG_PRINT_L1("Failed to check ringct signatures: mismatched pubkeys/mixRing size");
+            return false;
+          }
+        }
+
+        for (size_t n = 0; n < pubkeys.size(); ++n)
+        {
+          for (size_t m = 0; m < pubkeys[n].size(); ++m)
+          {
+            if (pubkeys[n][m].dest != rct::rct2pk(rv.mixRing[n][m].dest))
+            {
+              LOG_PRINT_L1("Failed to check ringct signatures: mismatched pubkey at vin " << n << ", index " << m);
+              return false;
+            }
+            if (pubkeys[n][m].mask != rct::rct2pk(rv.mixRing[n][m].mask))
+            {
+              LOG_PRINT_L1("Failed to check ringct signatures: mismatched commitment at vin " << n << ", index " << m);
+              return false;
+            }
+          }
+        }
+      }
+
+      if (rv.p.MGs.size() != tx.vin.size())
+      {
+        LOG_PRINT_L1("Failed to check ringct signatures: mismatched MGs/vin sizes");
+        return false;
+      }
+      for (size_t n = 0; n < tx.vin.size(); ++n)
+      {
+        if (memcmp(&boost::get<txin_to_key>(tx.vin[n]).k_image, &rv.p.MGs[n].II[0], 32))
+        {
+          LOG_PRINT_L1("Failed to check ringct signatures: mismatched key image");
+          return false;
+        }
+      }
+
+      if (!rct::verRctSimple(rv))
+      {
+        LOG_PRINT_L1("Failed to check ringct signatures!");
+        return false;
+      }
+      break;
+    }
+    case rct::RCTTypeFull: {
+      // check all this, either recontructed (so should really pass), or not
+      {
+        bool size_matches = true;
+        for (size_t i = 0; i < pubkeys.size(); ++i)
+          size_matches &= pubkeys[i].size() == rv.mixRing.size();
+        for (size_t i = 0; i < rv.mixRing.size(); ++i)
+          size_matches &= pubkeys.size() == rv.mixRing[i].size();
+        if (!size_matches)
+        {
+          LOG_PRINT_L1("Failed to check ringct signatures: mismatched pubkeys/mixRing size");
+          return false;
+        }
+
+        for (size_t n = 0; n < pubkeys.size(); ++n)
+        {
+          for (size_t m = 0; m < pubkeys[n].size(); ++m)
+          {
+            if (pubkeys[n][m].dest != rct::rct2pk(rv.mixRing[m][n].dest))
+            {
+              LOG_PRINT_L1("Failed to check ringct signatures: mismatched pubkey at vin " << n << ", index " << m);
+              return false;
+            }
+            if (pubkeys[n][m].mask != rct::rct2pk(rv.mixRing[m][n].mask))
+            {
+              LOG_PRINT_L1("Failed to check ringct signatures: mismatched commitment at vin " << n << ", index " << m);
+              return false;
+            }
+          }
+        }
+      }
+
+      if (rv.p.MGs.size() != 1)
+      {
+        LOG_PRINT_L1("Failed to check ringct signatures: Bad MGs size");
+        return false;
+      }
+      if (rv.p.MGs[0].II.size() != tx.vin.size())
+      {
+        LOG_PRINT_L1("Failed to check ringct signatures: mismatched II/vin sizes");
+        return false;
+      }
+      for (size_t n = 0; n < tx.vin.size(); ++n)
+      {
+        if (memcmp(&boost::get<txin_to_key>(tx.vin[n]).k_image, &rv.p.MGs[0].II[n], 32))
+        {
+          LOG_PRINT_L1("Failed to check ringct signatures: mismatched II/vin sizes");
+          return false;
+        }
+      }
+
+      if (!rct::verRct(rv))
+      {
+        LOG_PRINT_L1("Failed to check ringct signatures!");
+        return false;
+      }
+      break;
+    }
+    default:
+      LOG_PRINT_L1("Unsupported rct type: " << rv.type);
+      return false;
+    }
+  }
   return true;
 }
 
 //------------------------------------------------------------------
-void Blockchain::check_ring_signature(const crypto::hash &tx_prefix_hash, const crypto::key_image &key_image, const std::vector<crypto::public_key> &pubkeys, const std::vector<crypto::signature>& sig, uint64_t &result)
+void Blockchain::check_ring_signature(const crypto::hash &tx_prefix_hash, const crypto::key_image &key_image, const std::vector<rct::ctkey> &pubkeys, const std::vector<crypto::signature>& sig, uint64_t &result)
 {
   if (m_is_in_checkpoint_zone)
   {
@@ -2285,7 +2696,8 @@ void Blockchain::check_ring_signature(const crypto::hash &tx_prefix_hash, const
   std::vector<const crypto::public_key *> p_output_keys;
   for (auto &key : pubkeys)
   {
-    p_output_keys.push_back(&key);
+    // rct::key and crypto::public_key have the same structure, avoid object ctor/memcpy
+    p_output_keys.push_back(&(const crypto::public_key&)key.dest);
   }
 
   result = crypto::check_ring_signature(tx_prefix_hash, key_image, p_output_keys, sig.data()) ? 1 : 0;
@@ -2321,7 +2733,7 @@ bool Blockchain::is_tx_spendtime_unlocked(uint64_t unlock_time) const
 // This function locates all outputs associated with a given input (mixins)
 // and validates that they exist and are usable.  It also checks the ring
 // signature for each input.
-bool Blockchain::check_tx_input(const txin_to_key& txin, const crypto::hash& tx_prefix_hash, const std::vector<crypto::signature>& sig, std::vector<crypto::public_key> &output_keys, uint64_t* pmax_related_block_height)
+bool Blockchain::check_tx_input(size_t tx_version, const txin_to_key& txin, const crypto::hash& tx_prefix_hash, const std::vector<crypto::signature>& sig, const rct::rctSig &rct_signatures, std::vector<rct::ctkey> &output_keys, uint64_t* pmax_related_block_height)
 {
   LOG_PRINT_L3("Blockchain::" << __func__);
 
@@ -2331,13 +2743,13 @@ bool Blockchain::check_tx_input(const txin_to_key& txin, const crypto::hash& tx_
 
   struct outputs_visitor
   {
-    std::vector<crypto::public_key >& m_output_keys;
+    std::vector<rct::ctkey >& m_output_keys;
     const Blockchain& m_bch;
-    outputs_visitor(std::vector<crypto::public_key>& output_keys, const Blockchain& bch) :
+    outputs_visitor(std::vector<rct::ctkey>& output_keys, const Blockchain& bch) :
       m_output_keys(output_keys), m_bch(bch)
     {
     }
-    bool handle_output(uint64_t unlock_time, const crypto::public_key &pubkey)
+    bool handle_output(uint64_t unlock_time, const crypto::public_key &pubkey, const rct::key &commitment)
     {
       //check tx unlock time
       if (!m_bch.is_tx_spendtime_unlocked(unlock_time))
@@ -2351,7 +2763,7 @@ bool Blockchain::check_tx_input(const txin_to_key& txin, const crypto::hash& tx_
       // but only txout_to_key outputs are stored in the DB in the first place, done in
       // Blockchain*::add_output
 
-      m_output_keys.push_back(pubkey);
+      m_output_keys.push_back(rct::ctkey({rct::pk2rct(pubkey), commitment}));
       return true;
     }
   };
@@ -2360,7 +2772,7 @@ bool Blockchain::check_tx_input(const txin_to_key& txin, const crypto::hash& tx_
 
   // collect output keys
   outputs_visitor vi(output_keys, *this);
-  if (!scan_outputkeys_for_indexes(txin, vi, tx_prefix_hash, pmax_related_block_height))
+  if (!scan_outputkeys_for_indexes(tx_version, txin, vi, tx_prefix_hash, pmax_related_block_height))
   {
     LOG_PRINT_L1("Failed to get output keys for tx with amount = " << print_money(txin.amount) << " and count indexes " << txin.key_offsets.size());
     return false;
@@ -2371,7 +2783,10 @@ bool Blockchain::check_tx_input(const txin_to_key& txin, const crypto::hash& tx_
     LOG_PRINT_L1("Output keys for tx with amount = " << txin.amount << " and count indexes " << txin.key_offsets.size() << " returned wrong keys count " << output_keys.size());
     return false;
   }
+  if (tx_version == 1) {
     CHECK_AND_ASSERT_MES(sig.size() == output_keys.size(), false, "internal error: tx signatures count=" << sig.size() << " mismatch with outputs keys count for inputs=" << output_keys.size());
+  }
+  // rct_signatures will be expanded after this
   return true;
 }
 //------------------------------------------------------------------
@@ -2453,6 +2868,8 @@ void Blockchain::return_tx_to_pool(const std::vector<transaction> &txs)
 //------------------------------------------------------------------
 bool Blockchain::flush_txes_from_pool(const std::list<crypto::hash> &txids)
 {
+  CRITICAL_REGION_LOCAL(m_tx_pool);
+
   bool res = true;
   for (const auto &txid: txids)
   {
@@ -2461,7 +2878,7 @@ bool Blockchain::flush_txes_from_pool(const std::list<crypto::hash> &txids)
     uint64_t fee;
     bool relayed;
     LOG_PRINT_L1("Removing txid " << txid << " from the pool");
-    if(!m_tx_pool.take_tx(txid, tx, blob_size, fee, relayed))
+    if(m_tx_pool.have_tx(txid) && !m_tx_pool.take_tx(txid, tx, blob_size, fee, relayed))
     {
       LOG_PRINT_L0("Failed to remove txid " << txid << " from the pool");
       res = false;

src/cryptonote_core/blockchain.h

@@ -454,6 +454,23 @@ namespace cryptonote
      */
     bool get_outs(const COMMAND_RPC_GET_OUTPUTS::request& req, COMMAND_RPC_GET_OUTPUTS::response& res) const;
 
+    /**
+     * @brief gets random ringct outputs to mix with
+     *
+     * This function takes an RPC request for outputs to mix with
+     * and creates an RPC response with the resultant output indices
+     * and the matching keys.
+     *
+     * Outputs to mix with are randomly selected from the utxo set
+     * for each output amount in the request.
+     *
+     * @param req the output amounts and number of mixins to select
+     * @param res return-by-reference the resultant output indices
+     *
+     * @return true
+     */
+    bool get_random_rct_outs(const COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::request& req, COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::response& res) const;
+
     /**
      * @brief gets the global indices for outputs from a given transaction
      *
@@ -483,6 +500,7 @@ namespace cryptonote
      * validates a transaction's inputs as correctly used and not previously
      * spent.  also returns the hash and height of the most recent block
      * which contains an output that was used as an input to the transaction.
+     * The transaction's rct signatures, if any, are expanded.
      *
      * @param tx the transaction to validate
      * @param pmax_used_block_height return-by-reference block height of most recent input
@@ -492,7 +510,7 @@ namespace cryptonote
      *
      * @return false if any input is invalid, otherwise true
      */
-    bool check_tx_inputs(const transaction& tx, uint64_t& pmax_used_block_height, crypto::hash& max_used_block_id, tx_verification_context &tvc, bool kept_by_block = false);
+    bool check_tx_inputs(transaction& tx, uint64_t& pmax_used_block_height, crypto::hash& max_used_block_id, tx_verification_context &tvc, bool kept_by_block = false);
 
     /**
      * @brief check that a transaction's outputs conform to current standards
@@ -867,11 +885,12 @@ namespace cryptonote
      * @param vis an instance of the visitor to use
      * @param tx_prefix_hash the hash of the associated transaction_prefix
      * @param pmax_related_block_height return-by-pointer the height of the most recent block in the input set
+     * @param tx_version version of the tx, if > 1 we also get commitments
      *
      * @return false if any keys are not found or any inputs are not unlocked, otherwise true
      */
     template<class visitor_t>
-    inline bool scan_outputkeys_for_indexes(const txin_to_key& tx_in_to_key, visitor_t &vis, const crypto::hash &tx_prefix_hash, uint64_t* pmax_related_block_height = NULL) const;
+    inline bool scan_outputkeys_for_indexes(size_t tx_version, const txin_to_key& tx_in_to_key, visitor_t &vis, const crypto::hash &tx_prefix_hash, uint64_t* pmax_related_block_height = NULL) const;
 
     /**
      * @brief collect output public keys of a transaction input set
@@ -883,15 +902,17 @@ namespace cryptonote
      * If pmax_related_block_height is not NULL, its value is set to the height
      * of the most recent block which contains an output used in the input set
      *
+     * @param tx_version the transaction version
      * @param txin the transaction input
      * @param tx_prefix_hash the transaction prefix hash, for caching organization
      * @param sig the input signature
      * @param output_keys return-by-reference the public keys of the outputs in the input set
+     * @param rct_signatures the ringCT signatures, which are only valid if tx version > 1
      * @param pmax_related_block_height return-by-pointer the height of the most recent block in the input set
      *
      * @return false if any output is not yet unlocked, or is missing, otherwise true
      */
-    bool check_tx_input(const txin_to_key& txin, const crypto::hash& tx_prefix_hash, const std::vector<crypto::signature>& sig, std::vector<crypto::public_key> &output_keys, uint64_t* pmax_related_block_height);
+    bool check_tx_input(size_t tx_version,const txin_to_key& txin, const crypto::hash& tx_prefix_hash, const std::vector<crypto::signature>& sig, const rct::rctSig &rct_signatures, std::vector<rct::ctkey> &output_keys, uint64_t* pmax_related_block_height);
 
     /**
      * @brief validate a transaction's inputs and their keys
@@ -899,6 +920,7 @@ namespace cryptonote
      * This function validates transaction inputs and their keys.  Previously
      * it also performed double spend checking, but that has been moved to its
      * own function.
+     * The transaction's rct signatures, if any, are expanded.
      *
      * If pmax_related_block_height is not NULL, its value is set to the height
      * of the most recent block which contains an output used in any input set
@@ -912,7 +934,7 @@ namespace cryptonote
      *
      * @return false if any validation step fails, otherwise true
      */
-    bool check_tx_inputs(const transaction& tx, tx_verification_context &tvc, uint64_t* pmax_used_block_height = NULL);
+    bool check_tx_inputs(transaction& tx, tx_verification_context &tvc, uint64_t* pmax_used_block_height = NULL);
 
     /**
      * @brief performs a blockchain reorganization according to the longest chain rule
@@ -1053,6 +1075,15 @@ namespace cryptonote
      */
     void add_out_to_get_random_outs(COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::outs_for_amount& result_outs, uint64_t amount, size_t i) const;
 
+    /**
+     * @brief adds the given output to the requested set of random ringct outputs
+     *
+     * @param outs return-by-reference the set the output is to be added to
+     * @param amount the output amount (0 for rct inputs)
+     * @param i the rct output index
+     */
+    void add_out_to_get_rct_random_outs(std::list<COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::out_entry>& outs, uint64_t amount, size_t i) const;
+
     /**
      * @brief checks if a transaction is unlocked (its outputs spendable)
      *
@@ -1172,7 +1203,7 @@ namespace cryptonote
      * @param result false if the ring signature is invalid, otherwise true
      */
     void check_ring_signature(const crypto::hash &tx_prefix_hash, const crypto::key_image &key_image,
-        const std::vector<crypto::public_key> &pubkeys, const std::vector<crypto::signature> &sig, uint64_t &result);
+        const std::vector<rct::ctkey> &pubkeys, const std::vector<crypto::signature> &sig, uint64_t &result);
 
     /**
      * @brief loads block hashes from compiled-in data set
@@ -1182,5 +1213,14 @@ namespace cryptonote
      * a useful state.
      */
     void load_compiled_in_block_hashes();
+
+    /**
+     * @brief expands v2 transaction data from blockchain
+     *
+     * RingCT transactions do not transmit some of their data if it
+     * can be reconstituted by the receiver. This function expands
+     * that implicit data.
+     */
+    bool expand_transaction_2(transaction &tx, const crypto::hash &tx_prefix_hash, const std::vector<std::vector<rct::ctkey>> &pubkeys);
   };
 }  // namespace cryptonote

src/cryptonote_core/blockchain_storage.h

@@ -1,377 +0,0 @@
-// Copyright (c) 2014-2016, The Monero Project
-// 
-// All rights reserved.
-// 
-// Redistribution and use in source and binary forms, with or without modification, are
-// permitted provided that the following conditions are met:
-// 
-// 1. Redistributions of source code must retain the above copyright notice, this list of
-//    conditions and the following disclaimer.
-// 
-// 2. Redistributions in binary form must reproduce the above copyright notice, this list
-//    of conditions and the following disclaimer in the documentation and/or other
-//    materials provided with the distribution.
-// 
-// 3. Neither the name of the copyright holder nor the names of its contributors may be
-//    used to endorse or promote products derived from this software without specific
-//    prior written permission.
-// 
-// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
-// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
-// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
-// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
-// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
-// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-// 
-// Parts of this file are originally copyright (c) 2012-2013 The Cryptonote developers
-
-#pragma once
-#include <boost/serialization/serialization.hpp>
-#include <boost/serialization/version.hpp>
-#include <boost/serialization/list.hpp>
-#include <boost/multi_index_container.hpp>
-#include <boost/multi_index/global_fun.hpp>
-#include <boost/multi_index/hashed_index.hpp>
-#include <boost/multi_index/member.hpp>
-#include <boost/foreach.hpp>
-#include <atomic>
-
-#include "syncobj.h"
-#include "string_tools.h"
-#include "tx_pool.h"
-#include "cryptonote_basic.h"
-#include "common/util.h"
-#include "cryptonote_protocol/cryptonote_protocol_defs.h"
-#include "rpc/core_rpc_server_commands_defs.h"
-#include "difficulty.h"
-#include "cryptonote_core/cryptonote_format_utils.h"
-#include "verification_context.h"
-#include "crypto/hash.h"
-#include "checkpoints.h"
-
-namespace cryptonote
-{
-
-  /************************************************************************/
-  /*                                                                      */
-  /************************************************************************/
-  class blockchain_storage
-  {
-  public:
-    struct transaction_chain_entry
-    {
-      transaction tx;
-      uint64_t m_keeper_block_height;
-      size_t m_blob_size;
-      std::vector<uint64_t> m_global_output_indexes;
-    };
-
-    struct block_extended_info
-    {
-      block   bl;
-      uint64_t height;
-      size_t block_cumulative_size;
-      difficulty_type cumulative_difficulty;
-      uint64_t already_generated_coins;
-    };
-
-    blockchain_storage(tx_memory_pool* tx_pool):m_tx_pool(tx_pool), m_current_block_cumul_sz_limit(0), m_is_in_checkpoint_zone(false), m_is_blockchain_storing(false), m_enforce_dns_checkpoints(false)
-    {};
-
-    bool init() { return init(tools::get_default_data_dir(), true); }
-    bool init(const std::string& config_folder, bool testnet = false);
-    bool deinit();
-
-    void set_checkpoints(checkpoints&& chk_pts) { m_checkpoints = chk_pts; }
-
-    //bool push_new_block();
-    bool get_blocks(uint64_t start_offset, size_t count, std::list<block>& blocks, std::list<transaction>& txs) const;
-    bool get_blocks(uint64_t start_offset, size_t count, std::list<block>& blocks) const;
-    bool get_alternative_blocks(std::list<block>& blocks) const;
-    size_t get_alternative_blocks_count() const;
-    crypto::hash get_block_id_by_height(uint64_t height) const;
-    bool get_block_by_hash(const crypto::hash &h, block &blk) const;
-    void get_all_known_block_ids(std::list<crypto::hash> &main, std::list<crypto::hash> &alt, std::list<crypto::hash> &invalid) const;
-
-    template<class archive_t>
-    void serialize(archive_t & ar, const unsigned int version);
-
-    bool have_tx(const crypto::hash &id) const;
-    bool have_tx_keyimges_as_spent(const transaction &tx) const;
-    bool have_tx_keyimg_as_spent(const crypto::key_image &key_im) const;
-    const transaction *get_tx(const crypto::hash &id) const;
-
-    uint64_t get_current_blockchain_height() const;
-    crypto::hash get_tail_id() const;
-    crypto::hash get_tail_id(uint64_t& height) const;
-    difficulty_type get_difficulty_for_next_block() const;
-    bool add_new_block(const block& bl_, block_verification_context& bvc);
-    bool reset_and_set_genesis_block(const block& b);
-    bool create_block_template(block& b, const account_public_address& miner_address, difficulty_type& di, uint64_t& height, const blobdata& ex_nonce) const;
-    bool have_block(const crypto::hash& id) const;
-    size_t get_total_transactions() const;
-    bool get_outs(uint64_t amount, std::list<crypto::public_key>& pkeys) const;
-    bool get_short_chain_history(std::list<crypto::hash>& ids) const;
-    bool find_blockchain_supplement(const std::list<crypto::hash>& qblock_ids, NOTIFY_RESPONSE_CHAIN_ENTRY::request& resp) const;
-    bool find_blockchain_supplement(const std::list<crypto::hash>& qblock_ids, uint64_t& starter_offset) const;
-    bool find_blockchain_supplement(const uint64_t req_start_block, const std::list<crypto::hash>& qblock_ids, std::list<std::pair<block, std::list<transaction> > >& blocks, uint64_t& total_height, uint64_t& start_height, size_t max_count) const;
-    bool handle_get_objects(NOTIFY_REQUEST_GET_OBJECTS::request& arg, NOTIFY_RESPONSE_GET_OBJECTS::request& rsp);
-    bool handle_get_objects(const COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::request& req, COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::response& res);
-    bool get_random_outs_for_amounts(const COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::request& req, COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::response& res) const;
-    bool get_backward_blocks_sizes(size_t from_height, std::vector<size_t>& sz, size_t count) const;
-    bool get_tx_outputs_gindexs(const crypto::hash& tx_id, std::vector<uint64_t>& indexs) const;
-    bool store_blockchain();
-
-    bool check_tx_inputs(const transaction& tx, uint64_t& pmax_used_block_height, crypto::hash& max_used_block_id) const;
-    uint64_t get_current_cumulative_blocksize_limit() const;
-    bool is_storing_blockchain()const{return m_is_blockchain_storing;}
-    uint64_t block_difficulty(size_t i) const;
-    double get_avg_block_size( size_t count) const;
-    bool flush_txes_from_pool(const std::list<crypto::hash> &txids);
-
-    template<class t_ids_container, class t_blocks_container, class t_missed_container>
-    bool get_blocks(const t_ids_container& block_ids, t_blocks_container& blocks, t_missed_container& missed_bs) const
-    {
-      CRITICAL_REGION_LOCAL(m_blockchain_lock);
-
-      BOOST_FOREACH(const auto& bl_id, block_ids)
-      {
-        auto it = m_blocks_index.find(bl_id);
-        if(it == m_blocks_index.end())
-          missed_bs.push_back(bl_id);
-        else
-        {
-          CHECK_AND_ASSERT_MES(it->second < m_blocks.size(), false, "Internal error: bl_id=" << epee::string_tools::pod_to_hex(bl_id)
-            << " have index record with offset="<<it->second<< ", bigger then m_blocks.size()=" << m_blocks.size());
-            blocks.push_back(m_blocks[it->second].bl);
-        }
-      }
-      return true;
-    }
-
-    template<class t_ids_container, class t_tx_container, class t_missed_container>
-    bool get_transactions(const t_ids_container& txs_ids, t_tx_container& txs, t_missed_container& missed_txs) const
-    {
-      CRITICAL_REGION_LOCAL(m_blockchain_lock);
-
-      BOOST_FOREACH(const auto& tx_id, txs_ids)
-      {
-        auto it = m_transactions.find(tx_id);
-        if(it == m_transactions.end())
-        {
-          transaction tx;
-          if(!m_tx_pool->get_transaction(tx_id, tx))
-            missed_txs.push_back(tx_id);
-          else
-            txs.push_back(tx);
-        }
-        else
-          txs.push_back(it->second.tx);
-      }
-      return true;
-    }
-    //debug functions
-    void print_blockchain(uint64_t start_index, uint64_t end_index) const;
-    void print_blockchain_index() const;
-    void print_blockchain_outs(const std::string& file) const;
-    void check_against_checkpoints(const checkpoints& points, bool enforce);
-    bool update_checkpoints(const std::string& file_path, bool check_dns);
-    void set_enforce_dns_checkpoints(bool enforce_checkpoints);
-
-    block get_block(uint64_t height) const { return m_blocks[height].bl; }
-    size_t get_block_size(uint64_t height) const { return m_blocks[height].block_cumulative_size; }
-    difficulty_type get_block_cumulative_difficulty(uint64_t height) const { return m_blocks[height].cumulative_difficulty; }
-    uint64_t get_block_coins_generated(uint64_t height) const { return m_blocks[height].already_generated_coins; }
-
-    bool for_all_key_images(std::function<bool(const crypto::key_image&)>) const;
-    bool for_all_blocks(std::function<bool(uint64_t height, const crypto::hash&, const block&)>) const;
-    bool for_all_transactions(std::function<bool(const crypto::hash&, const transaction&)>) const;
-    bool for_all_outputs(std::function<bool(uint64_t amount, const crypto::hash &tx_hash, size_t tx_idx)>) const;
-
-    // use for testing only
-    bool debug_pop_block_from_blockchain() { return pop_block_from_blockchain(); }
-
-  private:
-    typedef std::unordered_map<crypto::hash, size_t> blocks_by_id_index;
-    typedef std::unordered_map<crypto::hash, transaction_chain_entry> transactions_container;
-    typedef std::unordered_set<crypto::key_image> key_images_container;
-    typedef std::vector<block_extended_info> blocks_container;
-    typedef std::unordered_map<crypto::hash, block_extended_info> blocks_ext_by_hash;
-    typedef std::unordered_map<crypto::hash, block> blocks_by_hash;
-    typedef std::map<uint64_t, std::vector<std::pair<crypto::hash, size_t>>> outputs_container; //crypto::hash - tx hash, size_t - index of out in transaction
-
-    tx_memory_pool* m_tx_pool;
-    mutable epee::critical_section m_blockchain_lock; // TODO: add here reader/writer lock
-
-    // main chain
-    blocks_container m_blocks;               // height  -> block_extended_info
-    blocks_by_id_index m_blocks_index;       // crypto::hash -> height
-    transactions_container m_transactions;
-    key_images_container m_spent_keys;
-    size_t m_current_block_cumul_sz_limit;
-
-
-    // all alternative chains
-    blocks_ext_by_hash m_alternative_chains; // crypto::hash -> block_extended_info
-
-    // some invalid blocks
-    blocks_ext_by_hash m_invalid_blocks;     // crypto::hash -> block_extended_info
-    outputs_container m_outputs;
-
-
-    std::string m_config_folder;
-    checkpoints m_checkpoints;
-    std::atomic<bool> m_is_in_checkpoint_zone;
-    std::atomic<bool> m_is_blockchain_storing;
-
-    bool m_enforce_dns_checkpoints;
-    bool m_testnet;
-
-    // made private for consistency with blockchain.h
-    template<class visitor_t>
-    bool scan_outputkeys_for_indexes(const txin_to_key& tx_in_to_key, visitor_t& vis, uint64_t* pmax_related_block_height = NULL) const;
-    bool check_tx_input(const txin_to_key& txin, const crypto::hash& tx_prefix_hash, const std::vector<crypto::signature>& sig, uint64_t* pmax_related_block_height = NULL) const;
-    bool check_tx_inputs(const transaction& tx, const crypto::hash& tx_prefix_hash, uint64_t* pmax_used_block_height = NULL) const;
-    bool check_tx_inputs(const transaction& tx, uint64_t* pmax_used_block_height = NULL) const;
-
-    bool switch_to_alternative_blockchain(std::list<blocks_ext_by_hash::iterator>& alt_chain, bool discard_disconnected_chain);
-    bool pop_block_from_blockchain();
-    bool purge_block_data_from_blockchain(const block& b, size_t processed_tx_count);
-    bool purge_transaction_from_blockchain(const crypto::hash& tx_id);
-    bool purge_transaction_keyimages_from_blockchain(const transaction& tx, bool strict_check);
-
-    bool handle_block_to_main_chain(const block& bl, block_verification_context& bvc);
-    bool handle_block_to_main_chain(const block& bl, const crypto::hash& id, block_verification_context& bvc);
-    bool handle_alternative_block(const block& b, const crypto::hash& id, block_verification_context& bvc);
-    difficulty_type get_next_difficulty_for_alternative_chain(const std::list<blocks_ext_by_hash::iterator>& alt_chain, block_extended_info& bei) const;
-    bool prevalidate_miner_transaction(const block& b, uint64_t height) const;
-    bool validate_miner_transaction(const block& b, size_t cumulative_block_size, uint64_t fee, uint64_t& base_reward, uint64_t already_generated_coins) const;
-    bool validate_transaction(const block& b, uint64_t height, const transaction& tx) const;
-    bool rollback_blockchain_switching(std::list<block>& original_chain, size_t rollback_height);
-    bool add_transaction_from_block(const transaction& tx, const crypto::hash& tx_id, const crypto::hash& bl_id, uint64_t bl_height, size_t blob_size);
-    bool push_transaction_to_global_outs_index(const transaction& tx, const crypto::hash& tx_id, std::vector<uint64_t>& global_indexes);
-    bool pop_transaction_from_global_index(const transaction& tx, const crypto::hash& tx_id);
-    bool get_last_n_blocks_sizes(std::vector<size_t>& sz, size_t count) const;
-    bool add_out_to_get_random_outs(const std::vector<std::pair<crypto::hash, size_t> >& amount_outs, COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::outs_for_amount& result_outs, uint64_t amount, size_t i) const;
-    bool is_tx_spendtime_unlocked(uint64_t unlock_time) const;
-    bool add_block_as_invalid(const block& bl, const crypto::hash& h);
-    bool add_block_as_invalid(const block_extended_info& bei, const crypto::hash& h);
-    size_t find_end_of_allowed_index(const std::vector<std::pair<crypto::hash, size_t> >& amount_outs) const;
-    bool check_block_timestamp_main(const block& b) const;
-    bool check_block_timestamp(std::vector<uint64_t> timestamps, const block& b) const;
-    uint64_t get_adjusted_time() const;
-    bool complete_timestamps_vector(uint64_t start_height, std::vector<uint64_t>& timestamps) const;
-    bool update_next_comulative_size_limit();
-    bool store_genesis_block(bool testnet, bool check_added = false);
-  };
-
-
-  /************************************************************************/
-  /*                                                                      */
-  /************************************************************************/
-
-  #define CURRENT_BLOCKCHAIN_STORAGE_ARCHIVE_VER    12
-
-  template<class archive_t>
-  void blockchain_storage::serialize(archive_t & ar, const unsigned int version)
-  {
-    if(version < 11)
-      return;
-    CRITICAL_REGION_LOCAL(m_blockchain_lock);
-    ar & m_blocks;
-    ar & m_blocks_index;
-    ar & m_transactions;
-    ar & m_spent_keys;
-    ar & m_alternative_chains;
-    ar & m_outputs;
-    ar & m_invalid_blocks;
-    ar & m_current_block_cumul_sz_limit;
-    /*serialization bug workaround*/
-    if(version > 11)
-    {
-      uint64_t total_check_count = m_blocks.size() + m_blocks_index.size() + m_transactions.size() + m_spent_keys.size() + m_alternative_chains.size() + m_outputs.size() + m_invalid_blocks.size() + m_current_block_cumul_sz_limit;
-      if(archive_t::is_saving::value)
-      {        
-        ar & total_check_count;
-      }else
-      {
-        uint64_t total_check_count_loaded = 0;
-        ar & total_check_count_loaded;
-        if(total_check_count != total_check_count_loaded)
-        {
-          LOG_ERROR("Blockchain storage data corruption detected. total_count loaded from file = " << total_check_count_loaded << ", expected = " << total_check_count);
-
-          LOG_PRINT_L0("Blockchain storage:" << ENDL << 
-            "m_blocks: " << m_blocks.size() << ENDL  << 
-            "m_blocks_index: " << m_blocks_index.size() << ENDL  << 
-            "m_transactions: " << m_transactions.size() << ENDL  << 
-            "m_spent_keys: " << m_spent_keys.size() << ENDL  << 
-            "m_alternative_chains: " << m_alternative_chains.size() << ENDL  << 
-            "m_outputs: " << m_outputs.size() << ENDL  << 
-            "m_invalid_blocks: " << m_invalid_blocks.size() << ENDL  << 
-            "m_current_block_cumul_sz_limit: " << m_current_block_cumul_sz_limit);
-
-          throw std::runtime_error("Blockchain data corruption");
-        }
-      }
-    }
-
-
-    LOG_PRINT_L2("Blockchain storage:" << ENDL << 
-        "m_blocks: " << m_blocks.size() << ENDL  << 
-        "m_blocks_index: " << m_blocks_index.size() << ENDL  << 
-        "m_transactions: " << m_transactions.size() << ENDL  << 
-        "m_spent_keys: " << m_spent_keys.size() << ENDL  << 
-        "m_alternative_chains: " << m_alternative_chains.size() << ENDL  << 
-        "m_outputs: " << m_outputs.size() << ENDL  << 
-        "m_invalid_blocks: " << m_invalid_blocks.size() << ENDL  << 
-        "m_current_block_cumul_sz_limit: " << m_current_block_cumul_sz_limit);
-  }
-
-  //------------------------------------------------------------------
-  template<class visitor_t>
-  bool blockchain_storage::scan_outputkeys_for_indexes(const txin_to_key& tx_in_to_key, visitor_t& vis, uint64_t* pmax_related_block_height) const
-  {
-    CRITICAL_REGION_LOCAL(m_blockchain_lock);
-    auto it = m_outputs.find(tx_in_to_key.amount);
-    if(it == m_outputs.end() || !tx_in_to_key.key_offsets.size())
-      return false;
-
-    std::vector<uint64_t> absolute_offsets = relative_output_offsets_to_absolute(tx_in_to_key.key_offsets);
-
-
-    const std::vector<std::pair<crypto::hash, size_t> >& amount_outs_vec = it->second;
-    size_t count = 0;
-    BOOST_FOREACH(uint64_t i, absolute_offsets)
-    {
-      if(i >= amount_outs_vec.size() )
-      {
-        LOG_PRINT_L0("Wrong index in transaction inputs: " << i << ", expected maximum " << amount_outs_vec.size() - 1);
-        return false;
-      }
-      transactions_container::const_iterator tx_it = m_transactions.find(amount_outs_vec[i].first);
-      CHECK_AND_ASSERT_MES(tx_it != m_transactions.end(), false, "Wrong transaction id in output indexes: " << epee::string_tools::pod_to_hex(amount_outs_vec[i].first));
-      CHECK_AND_ASSERT_MES(amount_outs_vec[i].second < tx_it->second.tx.vout.size(), false,
-        "Wrong index in transaction outputs: " << amount_outs_vec[i].second << ", expected less then " << tx_it->second.tx.vout.size());
-      if(!vis.handle_output(tx_it->second.tx, tx_it->second.tx.vout[amount_outs_vec[i].second]))
-      {
-        LOG_PRINT_L0("Failed to handle_output for output no = " << count << ", with absolute offset " << i);
-        return false;
-      }
-      if(count++ == absolute_offsets.size()-1 && pmax_related_block_height)
-      {
-        if(*pmax_related_block_height < tx_it->second.m_keeper_block_height)
-          *pmax_related_block_height = tx_it->second.m_keeper_block_height;
-      }
-    }
-
-    return true;
-  }
-}
-
-
-
-BOOST_CLASS_VERSION(cryptonote::blockchain_storage, CURRENT_BLOCKCHAIN_STORAGE_ARCHIVE_VER)

src/cryptonote_core/cryptonote_basic.h

@@ -49,9 +49,7 @@
 #include "crypto/hash.h"
 #include "misc_language.h"
 #include "tx_extra.h"
-
+#include "ringct/rctTypes.h"
-#define DB_MEMORY 1
-#define DB_LMDB   2
 
 namespace cryptonote
 {
@@ -175,14 +173,14 @@ namespace cryptonote
 
     BEGIN_SERIALIZE()
       VARINT_FIELD(version)
-      if(CURRENT_TRANSACTION_VERSION < version) return false;
+      if(version == 0 || CURRENT_TRANSACTION_VERSION < version) return false;
       VARINT_FIELD(unlock_time)
       FIELD(vin)
       FIELD(vout)
       FIELD(extra)
     END_SERIALIZE()
 
-  protected:
+  public:
     transaction_prefix(){}
   };
 
@@ -190,6 +188,7 @@ namespace cryptonote
   {
   public:
     std::vector<std::vector<crypto::signature> > signatures; //count signatures  always the same as inputs count
+    rct::rctSig rct_signatures;
 
     transaction();
     virtual ~transaction();
@@ -198,6 +197,8 @@ namespace cryptonote
     BEGIN_SERIALIZE_OBJECT()
       FIELDS(*static_cast<transaction_prefix *>(this))
 
+      if (version == 1)
+      {
         ar.tag("signatures");
         ar.begin_array();
         PREPARE_CUSTOM_VECTOR_SERIALIZATION(vin.size(), signatures);
@@ -226,6 +227,29 @@ namespace cryptonote
             ar.delimit_array();
         }
         ar.end_array();
+      }
+      else
+      {
+        FIELD(rct_signatures)
+        switch (rct_signatures.type)
+        {
+        case rct::RCTTypeNull:
+          break;
+        case rct::RCTTypeSimple:
+          if (rct_signatures.mixRing.size() && rct_signatures.mixRing.size() != vin.size())
+            return false;
+          break;
+        case rct::RCTTypeFull:
+          for (size_t i = 0; i < rct_signatures.mixRing.size(); ++i)
+          {
+            if (rct_signatures.mixRing[i].size() != vin.size())
+              return false;
+          }
+          break;
+        default:
+          return false;
+        }
+      }
     END_SERIALIZE()
 
   private:
@@ -248,12 +272,13 @@ namespace cryptonote
   inline
   void transaction::set_null()
   {
-    version = 0;
+    version = 1;
     unlock_time = 0;
     vin.clear();
     vout.clear();
     extra.clear();
     signatures.clear();
+    rct_signatures.type = rct::RCTTypeNull;
   }
 
   inline

src/cryptonote_core/cryptonote_boost_serialization.h

@@ -37,9 +37,13 @@
 #include <boost/serialization/map.hpp>
 #include <boost/foreach.hpp>
 #include <boost/serialization/is_bitwise_serializable.hpp>
+#include <boost/archive/binary_iarchive.hpp>
+#include <boost/archive/binary_oarchive.hpp>
 #include "cryptonote_basic.h"
 #include "common/unordered_containers_boost_serialization.h"
 #include "crypto/crypto.h"
+#include "ringct/rctTypes.h"
+#include "ringct/rctOps.h"
 
 //namespace cryptonote {
 namespace boost
@@ -139,6 +143,16 @@ namespace boost
   }
 
 
+  template <class Archive>
+  inline void serialize(Archive &a, cryptonote::transaction_prefix &x, const boost::serialization::version_type ver)
+  {
+    a & x.version;
+    a & x.unlock_time;
+    a & x.vin;
+    a & x.vout;
+    a & x.extra;
+  }
+
   template <class Archive>
   inline void serialize(Archive &a, cryptonote::transaction &x, const boost::serialization::version_type ver)
   {
@@ -147,7 +161,10 @@ namespace boost
     a & x.vin;
     a & x.vout;
     a & x.extra;
+    if (x.version == 1)
       a & x.signatures;
+    else
+      a & x.rct_signatures;
   }
 
 
@@ -163,6 +180,107 @@ namespace boost
     a & b.miner_tx;
     a & b.tx_hashes;
   }
+
+  template <class Archive>
+  inline void serialize(Archive &a, rct::key &x, const boost::serialization::version_type ver)
+  {
+    a & reinterpret_cast<char (&)[sizeof(rct::key)]>(x);
+  }
+
+  template <class Archive>
+  inline void serialize(Archive &a, rct::ctkey &x, const boost::serialization::version_type ver)
+  {
+    a & x.dest;
+    a & x.mask;
+  }
+
+  template <class Archive>
+  inline void serialize(Archive &a, rct::rangeSig &x, const boost::serialization::version_type ver)
+  {
+    a & x.asig;
+    a & x.Ci;
+  }
+
+  template <class Archive>
+  inline void serialize(Archive &a, rct::asnlSig &x, const boost::serialization::version_type ver)
+  {
+    a & x.L1;
+    a & x.s2;
+    a & x.s;
+  }
+
+  template <class Archive>
+  inline void serialize(Archive &a, rct::mgSig &x, const boost::serialization::version_type ver)
+  {
+    a & x.ss;
+    a & x.cc;
+    // a & x.II; // not serialized, we can recover it from the tx vin
+  }
+
+  template <class Archive>
+  inline void serialize(Archive &a, rct::ecdhTuple &x, const boost::serialization::version_type ver)
+  {
+    a & x.mask;
+    a & x.amount;
+    // a & x.senderPk; // not serialized, as we do not use it in monero currently
+  }
+
+  inline void serializeOutPk(boost::archive::binary_iarchive &a, rct::ctkeyV &outPk_, const boost::serialization::version_type ver)
+  {
+    rct::keyV outPk;
+    a & outPk;
+    outPk_.resize(outPk.size());
+    for (size_t n = 0; n < outPk_.size(); ++n)
+    {
+      outPk_[n].dest = rct::identity();
+      outPk_[n].mask = outPk[n];
+    }
+  }
+
+  inline void serializeOutPk(boost::archive::binary_oarchive &a, rct::ctkeyV &outPk_, const boost::serialization::version_type ver)
+  {
+    rct::keyV outPk(outPk_.size());
+    for (size_t n = 0; n < outPk_.size(); ++n)
+      outPk[n] = outPk_[n].mask;
+    a & outPk;
+  }
+
+  template <class Archive>
+  inline void serialize(Archive &a, rct::rctSigBase &x, const boost::serialization::version_type ver)
+  {
+    a & x.type;
+    if (x.type == rct::RCTTypeNull)
+      return;
+    if (x.type != rct::RCTTypeFull && x.type != rct::RCTTypeSimple)
+      throw boost::archive::archive_exception(boost::archive::archive_exception::other_exception, "Unsupported rct type");
+    // a & x.message; message is not serialized, as it can be reconstructed from the tx data
+    // a & x.mixRing; mixRing is not serialized, as it can be reconstructed from the offsets
+    if (x.type == rct::RCTTypeSimple)
+      a & x.pseudoOuts;
+    a & x.ecdhInfo;
+    serializeOutPk(a, x.outPk, ver);
+    a & x.txnFee;
+  }
+
+  template <class Archive>
+  inline void serialize(Archive &a, rct::rctSig &x, const boost::serialization::version_type ver)
+  {
+    a & x.type;
+    if (x.type == rct::RCTTypeNull)
+      return;
+    if (x.type != rct::RCTTypeFull && x.type != rct::RCTTypeSimple)
+      throw boost::archive::archive_exception(boost::archive::archive_exception::other_exception, "Unsupported rct type");
+    // a & x.message; message is not serialized, as it can be reconstructed from the tx data
+    // a & x.mixRing; mixRing is not serialized, as it can be reconstructed from the offsets
+    if (x.type == rct::RCTTypeSimple)
+      a & x.pseudoOuts;
+    a & x.ecdhInfo;
+    serializeOutPk(a, x.outPk, ver);
+    a & x.txnFee;
+    //--------------
+    a & x.p.rangeSigs;
+    a & x.p.MGs;
+  }
 }
 }
 

src/cryptonote_core/cryptonote_core.cpp

@@ -43,6 +43,7 @@ using namespace epee;
 #include "misc_language.h"
 #include <csignal>
 #include "cryptonote_core/checkpoints.h"
+#include "ringct/rctTypes.h"
 #include "blockchain_db/blockchain_db.h"
 #include "blockchain_db/lmdb/db_lmdb.h"
 #if defined(BERKELEY_DB)
@@ -57,11 +58,7 @@ namespace cryptonote
   //-----------------------------------------------------------------------------------------------
   core::core(i_cryptonote_protocol* pprotocol):
               m_mempool(m_blockchain_storage),
-#if BLOCKCHAIN_DB == DB_LMDB
               m_blockchain_storage(m_mempool),
-#else
-              m_blockchain_storage(&m_mempool),
-#endif
               m_miner(this),
               m_miner_address(boost::value_initialized<account_public_address>()),
               m_starter_message_showed(false),
@@ -257,7 +254,6 @@ namespace cryptonote
     r = m_mempool.init(m_fakechain ? std::string() : m_config_folder);
     CHECK_AND_ASSERT_MES(r, false, "Failed to initialize memory pool");
 
-#if BLOCKCHAIN_DB == DB_LMDB
     std::string db_type = command_line::get_arg(vm, command_line::arg_db_type);
     std::string db_sync_mode = command_line::get_arg(vm, command_line::arg_db_sync_mode);
     bool fast_sync = command_line::get_arg(vm, command_line::arg_fast_block_sync) != 0;
@@ -405,9 +401,6 @@ namespace cryptonote
 
     bool show_time_stats = command_line::get_arg(vm, command_line::arg_show_time_stats) != 0;
     m_blockchain_storage.set_show_time_stats(show_time_stats);
-#else
-    r = m_blockchain_storage.init(m_config_folder, m_testnet);
-#endif
     CHECK_AND_ASSERT_MES(r, false, "Failed to initialize blockchain storage");
 
     // load json & DNS checkpoints, and verify them
@@ -505,6 +498,15 @@ namespace cryptonote
     }
     //std::cout << "!"<< tx.vin.size() << std::endl;
 
+    uint8_t version = m_blockchain_storage.get_current_hard_fork_version();
+    const size_t max_tx_version = version == 1 ? 1 : 2;
+    if (tx.version == 0 || tx.version > max_tx_version)
+    {
+      // v2 is the latest one we know
+      tvc.m_verifivation_failed = true;
+      return false;
+    }
+
     if(!check_tx_syntax(tx))
     {
       LOG_PRINT_L1("WRONG TRANSACTION BLOB, Failed to check tx " << tx_hash << " syntax, rejected");
@@ -560,6 +562,14 @@ namespace cryptonote
       LOG_PRINT_RED_L1("tx with invalid outputs, rejected for tx id= " << get_transaction_hash(tx));
       return false;
     }
+    if (tx.version > 1)
+    {
+      if (tx.rct_signatures.outPk.size() != tx.vout.size())
+      {
+        LOG_PRINT_RED_L1("tx with mismatched vout/outPk count, rejected for tx id= " << get_transaction_hash(tx));
+        return false;
+      }
+    }
 
     if(!check_money_overflow(tx))
     {
@@ -567,6 +577,8 @@ namespace cryptonote
       return false;
     }
 
+    if (tx.version == 1)
+    {
       uint64_t amount_in = 0;
       get_inputs_money_amount(tx, amount_in);
       uint64_t amount_out = get_outs_money_amount(tx);
@@ -576,6 +588,8 @@ namespace cryptonote
         LOG_PRINT_RED_L1("tx with wrong amounts: ins " << amount_in << ", outs " << amount_out << ", rejected for tx id= " << get_transaction_hash(tx));
         return false;
       }
+    }
+    // for version > 1, ringct signatures check verifies amounts match
 
     if(!keeped_by_block && get_object_blobsize(tx) >= m_blockchain_storage.get_current_cumulative_blocksize_limit() - CRYPTONOTE_COINBASE_BLOB_RESERVED_SIZE)
     {
@@ -714,6 +728,11 @@ namespace cryptonote
     return m_blockchain_storage.get_outs(req, res);
   }
   //-----------------------------------------------------------------------------------------------
+  bool core::get_random_rct_outs(const COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::request& req, COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::response& res) const
+  {
+    return m_blockchain_storage.get_random_rct_outs(req, res);
+  }
+  //-----------------------------------------------------------------------------------------------
   bool core::get_tx_outputs_gindexs(const crypto::hash& tx_id, std::vector<uint64_t>& indexs) const
   {
     return m_blockchain_storage.get_tx_outputs_gindexs(tx_id, indexs);
@@ -780,18 +799,14 @@ namespace cryptonote
   //-----------------------------------------------------------------------------------------------
   bool core::prepare_handle_incoming_blocks(const std::list<block_complete_entry> &blocks)
   {
-#if BLOCKCHAIN_DB == DB_LMDB
     m_blockchain_storage.prepare_handle_incoming_blocks(blocks);
-#endif
     return true;
   }
 
   //-----------------------------------------------------------------------------------------------
   bool core::cleanup_handle_incoming_blocks(bool force_sync)
   {
-#if BLOCKCHAIN_DB == DB_LMDB
     m_blockchain_storage.cleanup_handle_incoming_blocks(force_sync);
-#endif
     return true;
   }
 
@@ -918,11 +933,6 @@ namespace cryptonote
       m_starter_message_showed = true;
     }
 
-#if BLOCKCHAIN_DB == DB_LMDB
-    // m_store_blockchain_interval.do_call(boost::bind(&Blockchain::store_blockchain, &m_blockchain_storage));
-#else
-    m_store_blockchain_interval.do_call(boost::bind(&blockchain_storage::store_blockchain, &m_blockchain_storage));
-#endif
     m_fork_moaner.do_call(boost::bind(&core::check_fork_time, this));
     m_txpool_auto_relayer.do_call(boost::bind(&core::relay_txpool_transactions, this));
     m_miner.on_idle();
@@ -932,7 +942,6 @@ namespace cryptonote
   //-----------------------------------------------------------------------------------------------
   bool core::check_fork_time()
   {
-#if BLOCKCHAIN_DB == DB_LMDB
     HardFork::State state = m_blockchain_storage.get_hard_fork_state();
     switch (state) {
       case HardFork::LikelyForked:
@@ -951,7 +960,6 @@ namespace cryptonote
       default:
         break;
     }
-#endif
     return true;
   }
   //-----------------------------------------------------------------------------------------------

src/cryptonote_core/cryptonote_core.h

@@ -40,11 +40,7 @@
 #include "cryptonote_protocol/cryptonote_protocol_handler_common.h"
 #include "storages/portable_storage_template_helper.h"
 #include "tx_pool.h"
-#if BLOCKCHAIN_DB == DB_LMDB
 #include "blockchain.h"
-#else
-#include "blockchain_storage.h"
-#endif
 #include "miner.h"
 #include "connection_context.h"
 #include "cryptonote_core/cryptonote_stat_info.h"
@@ -483,6 +479,14 @@ namespace cryptonote
       */
      bool get_outs(const COMMAND_RPC_GET_OUTPUTS::request& req, COMMAND_RPC_GET_OUTPUTS::response& res) const;
 
+     /**
+      *
+      * @copydoc Blockchain::get_random_rct_outs
+      *
+      * @note see Blockchain::get_random_rct_outs
+      */
+     bool get_random_rct_outs(const COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::request& req, COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::response& res) const;
+
 
      /**
       * @copydoc miner::pause
@@ -498,7 +502,6 @@ namespace cryptonote
       */
      void resume_mine();
 
-#if BLOCKCHAIN_DB == DB_LMDB
      /**
       * @brief gets the Blockchain instance
       *
@@ -512,10 +515,6 @@ namespace cryptonote
       * @return a const reference to the Blockchain instance
       */
      const Blockchain& get_blockchain_storage()const{return m_blockchain_storage;}
-#else
-     blockchain_storage& get_blockchain_storage(){return m_blockchain_storage;}
-     const blockchain_storage& get_blockchain_storage()const{return m_blockchain_storage;}
-#endif
 
      /**
       * @copydoc Blockchain::print_blockchain
@@ -765,11 +764,7 @@ namespace cryptonote
      uint64_t m_test_drop_download_height = 0; //!< height under which to drop incoming blocks, if doing so
 
      tx_memory_pool m_mempool; //!< transaction pool instance
-#if BLOCKCHAIN_DB == DB_LMDB
      Blockchain m_blockchain_storage; //!< Blockchain instance
-#else
-     blockchain_storage m_blockchain_storage;
-#endif
 
      i_cryptonote_protocol* m_pprotocol; //!< cryptonote protocol instance
 

src/cryptonote_core/cryptonote_format_utils.cpp

@@ -37,6 +37,7 @@ using namespace epee;
 #include "miner.h"
 #include "crypto/crypto.h"
 #include "crypto/hash.h"
+#include "ringct/rctSigs.h"
 
 #define ENCRYPTED_PAYMENT_ID_TAIL 0x8d
 
@@ -100,7 +101,7 @@ namespace cryptonote
     CHECK_AND_ASSERT_MES(r, false, "Failed to parse transaction from blob");
     //TODO: validate tx
 
-    crypto::cn_fast_hash(tx_blob.data(), tx_blob.size(), tx_hash);
+    get_transaction_hash(tx, tx_hash);
     get_transaction_prefix_hash(tx, tx_prefix_hash);
     return true;
   }
@@ -135,7 +136,10 @@ namespace cryptonote
     // from hard fork 2, we cut out the low significant digits. This makes the tx smaller, and
     // keeps the paid amount almost the same. The unpaid remainder gets pushed back to the
     // emission schedule
-    if (hard_fork_version >= 2) {
+    // from hard fork 4, we use a single "dusty" output. This makes the tx even smaller,
+    // and avoids the quantization. These outputs will be added as rct outputs with identity
+    // masks, to they can be used as rct inputs.
+    if (hard_fork_version >= 2 && hard_fork_version < 4) {
       block_reward = block_reward - block_reward % ::config::BASE_REWARD_CLAMP_THRESHOLD;
     }
 
@@ -145,12 +149,16 @@ namespace cryptonote
       [&out_amounts](uint64_t a_dust) { out_amounts.push_back(a_dust); });
 
     CHECK_AND_ASSERT_MES(1 <= max_outs, false, "max_out must be non-zero");
-    if (height == 0)
+    if (height == 0 || hard_fork_version >= 4)
     {
       // the genesis block was not decomposed, for unknown reasons
       while (max_outs < out_amounts.size())
       {
-        out_amounts[out_amounts.size() - 2] += out_amounts.back();
+        //out_amounts[out_amounts.size() - 2] += out_amounts.back();
+        //out_amounts.resize(out_amounts.size() - 1);
+        out_amounts[1] += out_amounts[0];
+        for (size_t n = 1; n < out_amounts.size(); ++n)
+          out_amounts[n - 1] = out_amounts[n];
         out_amounts.resize(out_amounts.size() - 1);
       }
     }
@@ -181,7 +189,11 @@ namespace cryptonote
 
     CHECK_AND_ASSERT_MES(summary_amounts == block_reward, false, "Failed to construct miner tx, summary_amounts = " << summary_amounts << " not equal block_reward = " << block_reward);
 
-    tx.version = CURRENT_TRANSACTION_VERSION;
+    if (hard_fork_version >= 4)
+      tx.version = 2;
+    else
+      tx.version = 1;
+
     //lock
     tx.unlock_time = height + CRYPTONOTE_MINED_MONEY_UNLOCK_WINDOW;
     tx.vin.push_back(in);
@@ -252,6 +264,11 @@ namespace cryptonote
   //---------------------------------------------------------------
   bool get_tx_fee(const transaction& tx, uint64_t & fee)
   {
+    if (tx.version > 1)
+    {
+      fee = tx.rct_signatures.txnFee;
+      return true;
+    }
     uint64_t amount_in = 0;
     uint64_t amount_out = 0;
     BOOST_FOREACH(auto& in, tx.vin)
@@ -315,6 +332,11 @@ namespace cryptonote
     return pub_key_field.pub_key;
   }
   //---------------------------------------------------------------
+  crypto::public_key get_tx_pub_key_from_extra(const transaction_prefix& tx_prefix)
+  {
+    return get_tx_pub_key_from_extra(tx_prefix.extra);
+  }
+  //---------------------------------------------------------------
   crypto::public_key get_tx_pub_key_from_extra(const transaction& tx)
   {
     return get_tx_pub_key_from_extra(tx.extra);
@@ -447,13 +469,16 @@ namespace cryptonote
     return encrypt_payment_id(payment_id, public_key, secret_key);
   }
   //---------------------------------------------------------------
-  bool construct_tx_and_get_tx_key(const account_keys& sender_account_keys, const std::vector<tx_source_entry>& sources, const std::vector<tx_destination_entry>& destinations, std::vector<uint8_t> extra, transaction& tx, uint64_t unlock_time, crypto::secret_key &tx_key)
+  bool construct_tx_and_get_tx_key(const account_keys& sender_account_keys, const std::vector<tx_source_entry>& sources, const std::vector<tx_destination_entry>& destinations, std::vector<uint8_t> extra, transaction& tx, uint64_t unlock_time, crypto::secret_key &tx_key, bool rct)
   {
+    std::vector<crypto::secret_key> amount_keys;
     tx.vin.clear();
     tx.vout.clear();
     tx.signatures.clear();
+    tx.rct_signatures = rct::rctSig();
+    amount_keys.clear();
 
-    tx.version = CURRENT_TRANSACTION_VERSION;
+    tx.version = rct ? 2 : 1;
     tx.unlock_time = unlock_time;
 
     tx.extra = extra;
@@ -509,7 +534,6 @@ namespace cryptonote
     };
     std::vector<input_generation_context_data> in_contexts;
 
-
     uint64_t summary_inputs_money = 0;
     //fill inputs
     BOOST_FOREACH(const tx_source_entry& src_entr,  sources)
@@ -529,7 +553,7 @@ namespace cryptonote
         return false;
 
       //check that derivated key is equal with real output key
-      if( !(in_ephemeral.pub == src_entr.outputs[src_entr.real_output].second) )
+      if( !(in_ephemeral.pub == src_entr.outputs[src_entr.real_output].second.dest) )
       {
         LOG_ERROR("derived public key missmatch with output public key! "<< ENDL << "derived_key:"
           << string_tools::pod_to_hex(in_ephemeral.pub) << ENDL << "real output_public_key:"
@@ -559,12 +583,18 @@ namespace cryptonote
     size_t output_index = 0;
     BOOST_FOREACH(const tx_destination_entry& dst_entr,  shuffled_dsts)
     {
-      CHECK_AND_ASSERT_MES(dst_entr.amount > 0, false, "Destination with wrong amount: " << dst_entr.amount);
+      CHECK_AND_ASSERT_MES(dst_entr.amount > 0 || tx.version > 1, false, "Destination with wrong amount: " << dst_entr.amount);
       crypto::key_derivation derivation;
       crypto::public_key out_eph_public_key;
       bool r = crypto::generate_key_derivation(dst_entr.addr.m_view_public_key, txkey.sec, derivation);
       CHECK_AND_ASSERT_MES(r, false, "at creation outs: failed to generate_key_derivation(" << dst_entr.addr.m_view_public_key << ", " << txkey.sec << ")");
 
+      if (tx.version > 1)
+      {
+        crypto::secret_key scalar1;
+        crypto::derivation_to_scalar(derivation, output_index, scalar1);
+        amount_keys.push_back(scalar1);
+      }
       r = crypto::derive_public_key(derivation, output_index, dst_entr.addr.m_spend_public_key, out_eph_public_key);
       CHECK_AND_ASSERT_MES(r, false, "at creation outs: failed to derive_public_key(" << derivation << ", " << output_index << ", "<< dst_entr.addr.m_spend_public_key << ")");
 
@@ -586,6 +616,8 @@ namespace cryptonote
     }
 
 
+    if (tx.version == 1)
+    {
       //generate ring signatures
       crypto::hash tx_prefix_hash;
       get_transaction_prefix_hash(tx, tx_prefix_hash);
@@ -596,10 +628,14 @@ namespace cryptonote
       {
         ss_ring_s << "pub_keys:" << ENDL;
         std::vector<const crypto::public_key*> keys_ptrs;
+        std::vector<crypto::public_key> keys(src_entr.outputs.size());
+        size_t ii = 0;
         BOOST_FOREACH(const tx_source_entry::output_entry& o, src_entr.outputs)
         {
-        keys_ptrs.push_back(&o.second);
+          keys[ii] = rct2pk(o.second.dest);
-        ss_ring_s << o.second << ENDL;
+          keys_ptrs.push_back(&keys[ii]);
+          ss_ring_s << o.second.dest << ENDL;
+          ++ii;
         }
 
         tx.signatures.push_back(std::vector<crypto::signature>());
@@ -613,6 +649,115 @@ namespace cryptonote
       }
 
       LOG_PRINT2("construct_tx.log", "transaction_created: " << get_transaction_hash(tx) << ENDL << obj_to_json_str(tx) << ENDL << ss_ring_s.str() , LOG_LEVEL_3);
+    }
+    else
+    {
+      bool all_rct_inputs = true;
+      size_t n_total_outs = sources[0].outputs.size(); // only for non-simple rct
+      BOOST_FOREACH(const tx_source_entry& src_entr,  sources)
+        all_rct_inputs &= !(src_entr.mask == rct::identity());
+
+      // the non-simple version is slightly smaller, but assumes all real inputs
+      // are on the same index, so can only be used if there just one ring.
+      bool use_simple_rct = sources.size() > 1;
+
+      if (!use_simple_rct)
+      {
+        // non simple ringct requires all real inputs to be at the same index for all inputs
+        BOOST_FOREACH(const tx_source_entry& src_entr,  sources)
+        {
+          if(src_entr.real_output != sources.begin()->real_output)
+          {
+            LOG_ERROR("All inputs must have the same index for non-simple ringct");
+            return false;
+          }
+        }
+
+        // enforce same mixin for all outputs
+        for (size_t i = 1; i < sources.size(); ++i) {
+          if (n_total_outs != sources[i].outputs.size()) {
+            LOG_ERROR("Non-simple ringct transaction has varying mixin");
+            return false;
+          }
+        }
+      }
+
+      uint64_t amount_in = 0, amount_out = 0;
+      rct::ctkeyV inSk;
+      // mixRing indexing is done the other way round for simple
+      rct::ctkeyM mixRing(use_simple_rct ? sources.size() : n_total_outs);
+      rct::keyV destinations;
+      std::vector<uint64_t> inamounts, outamounts;
+      std::vector<unsigned int> index;
+      for (size_t i = 0; i < sources.size(); ++i)
+      {
+        rct::ctkey ctkey;
+        amount_in += sources[i].amount;
+        inamounts.push_back(sources[i].amount);
+        index.push_back(sources[i].real_output);
+        // inSk: (secret key, mask)
+        ctkey.dest = rct::sk2rct(in_contexts[i].in_ephemeral.sec);
+        ctkey.mask = sources[i].mask;
+        inSk.push_back(ctkey);
+        // inPk: (public key, commitment)
+        // will be done when filling in mixRing
+      }
+      for (size_t i = 0; i < tx.vout.size(); ++i)
+      {
+        destinations.push_back(rct::pk2rct(boost::get<txout_to_key>(tx.vout[i].target).key));
+        outamounts.push_back(tx.vout[i].amount);
+        amount_out += tx.vout[i].amount;
+      }
+
+      if (use_simple_rct)
+      {
+        // mixRing indexing is done the other way round for simple
+        for (size_t i = 0; i < sources.size(); ++i)
+        {
+          mixRing[i].resize(sources[i].outputs.size());
+          for (size_t n = 0; n < sources[i].outputs.size(); ++n)
+          {
+            mixRing[i][n] = sources[i].outputs[n].second;
+          }
+        }
+      }
+      else
+      {
+        for (size_t i = 0; i < n_total_outs; ++i) // same index assumption
+        {
+          mixRing[i].resize(sources.size());
+          for (size_t n = 0; n < sources.size(); ++n)
+          {
+            mixRing[i][n] = sources[n].outputs[i].second;
+          }
+        }
+      }
+
+      // fee
+      if (!use_simple_rct && amount_in > amount_out)
+        outamounts.push_back(amount_in - amount_out);
+
+      // zero out all amounts to mask rct outputs, real amounts are now encrypted
+      for (size_t i = 0; i < tx.vin.size(); ++i)
+      {
+        if (sources[i].rct)
+          boost::get<txin_to_key>(tx.vin[i]).amount = 0;
+      }
+      for (size_t i = 0; i < tx.vout.size(); ++i)
+        tx.vout[i].amount = 0;
+
+      crypto::hash tx_prefix_hash;
+      get_transaction_prefix_hash(tx, tx_prefix_hash);
+      rct::ctkeyV outSk;
+      if (use_simple_rct)
+        tx.rct_signatures = rct::genRctSimple(rct::hash2rct(tx_prefix_hash), inSk, destinations, inamounts, outamounts, amount_in - amount_out, mixRing, (const rct::keyV&)amount_keys, index, outSk);
+      else
+        tx.rct_signatures = rct::genRct(rct::hash2rct(tx_prefix_hash), inSk, destinations, outamounts, mixRing, (const rct::keyV&)amount_keys, sources[0].real_output, outSk); // same index assumption
+
+      CHECK_AND_ASSERT_MES(tx.vout.size() == outSk.size(), false, "outSk size does not match vout");
+
+      LOG_PRINT2("construct_tx.log", "transaction_created: " << get_transaction_hash(tx) << ENDL << obj_to_json_str(tx) << ENDL, LOG_LEVEL_3);
+    }
 
     return true;
   }
@@ -661,7 +806,10 @@ namespace cryptonote
         << out.target.type().name() << ", expected " << typeid(txout_to_key).name()
         << ", in transaction id=" << get_transaction_hash(tx));
 
-      CHECK_AND_NO_ASSERT_MES(0 < out.amount, false, "zero amount ouput in transaction id=" << get_transaction_hash(tx));
+      if (tx.version == 1)
+      {
+        CHECK_AND_NO_ASSERT_MES(0 < out.amount, false, "zero amount output in transaction id=" << get_transaction_hash(tx));
+      }
 
       if(!check_key(boost::get<txout_to_key>(out.target).key))
         return false;
@@ -776,20 +924,49 @@ namespace cryptonote
   crypto::hash get_transaction_hash(const transaction& t)
   {
     crypto::hash h = null_hash;
-    size_t blob_size = 0;
+    get_transaction_hash(t, h, NULL);
-    get_object_hash(t, h, blob_size);
     return h;
   }
   //---------------------------------------------------------------
   bool get_transaction_hash(const transaction& t, crypto::hash& res)
   {
-    size_t blob_size = 0;
+    return get_transaction_hash(t, res, NULL);
-    return get_object_hash(t, res, blob_size);
+  }
+  //---------------------------------------------------------------
+  bool get_transaction_hash(const transaction& t, crypto::hash& res, size_t* blob_size)
+  {
+    // v1 transactions hash the entire blob
+    if (t.version == 1)
+    {
+      size_t ignored_blob_size, &blob_size_ref = blob_size ? *blob_size : ignored_blob_size;
+      return get_object_hash(t, res, blob_size_ref);
+    }
+
+    // v2 transactions hash different parts together, than hash the set of those hashes
+    crypto::hash hashes[3];
+
+    // prefix
+    get_transaction_prefix_hash(t, hashes[0]);
+
+    // base rct data
+    get_blob_hash(t_serializable_object_to_blob((const rct::rctSigBase&)t.rct_signatures), hashes[1]);
+
+    // prunable rct data
+    get_blob_hash(t_serializable_object_to_blob(t.rct_signatures.p), hashes[2]);
+
+    // the tx hash is the hash of the 3 hashes
+    res = cn_fast_hash(hashes, sizeof(hashes));
+
+    // we still need the size
+    if (blob_size)
+      *blob_size = get_object_blobsize(t);
+
+    return true;
   }
   //---------------------------------------------------------------
   bool get_transaction_hash(const transaction& t, crypto::hash& res, size_t& blob_size)
   {
-    return get_object_hash(t, res, blob_size);
+    return get_transaction_hash(t, res, &blob_size);
   }
   //---------------------------------------------------------------
   blobdata get_block_hashing_blob(const block& b)

src/cryptonote_core/cryptonote_format_utils.h

@@ -35,6 +35,7 @@
 #include "include_base_utils.h"
 #include "crypto/crypto.h"
 #include "crypto/hash.h"
+#include "ringct/rctOps.h"
 
 
 namespace cryptonote
@@ -50,13 +51,17 @@ namespace cryptonote
 
   struct tx_source_entry
   {
-    typedef std::pair<uint64_t, crypto::public_key> output_entry;
+    typedef std::pair<uint64_t, rct::ctkey> output_entry;
 
-    std::vector<output_entry> outputs;  //index + key
+    std::vector<output_entry> outputs;  //index + key + optional ringct commitment
     size_t real_output;                 //index in outputs vector of real output_entry
     crypto::public_key real_out_tx_key; //incoming real tx public key
     size_t real_output_in_tx_index;     //index in transaction outputs vector
     uint64_t amount;                    //money
+    bool rct;                           //true if the output is rct
+    rct::key mask;                      //ringct amount mask
+
+    void push_output(uint64_t idx, const crypto::public_key &k, uint64_t amount) { outputs.push_back(std::make_pair(idx, rct::ctkey({rct::pk2rct(k), rct::zeroCommit(amount)}))); }
   };
 
   struct tx_destination_entry
@@ -70,7 +75,7 @@ namespace cryptonote
 
   //---------------------------------------------------------------
   bool construct_tx(const account_keys& sender_account_keys, const std::vector<tx_source_entry>& sources, const std::vector<tx_destination_entry>& destinations, std::vector<uint8_t> extra, transaction& tx, uint64_t unlock_time);
-  bool construct_tx_and_get_tx_key(const account_keys& sender_account_keys, const std::vector<tx_source_entry>& sources, const std::vector<tx_destination_entry>& destinations, std::vector<uint8_t> extra, transaction& tx, uint64_t unlock_time, crypto::secret_key &txkey);
+  bool construct_tx_and_get_tx_key(const account_keys& sender_account_keys, const std::vector<tx_source_entry>& sources, const std::vector<tx_destination_entry>& destinations, std::vector<uint8_t> extra, transaction& tx, uint64_t unlock_time, crypto::secret_key &tx_key, bool rct = false);
 
   template<typename T>
   bool find_tx_extra_field_by_type(const std::vector<tx_extra_field>& tx_extra_fields, T& field)
@@ -85,6 +90,7 @@ namespace cryptonote
 
   bool parse_tx_extra(const std::vector<uint8_t>& tx_extra, std::vector<tx_extra_field>& tx_extra_fields);
   crypto::public_key get_tx_pub_key_from_extra(const std::vector<uint8_t>& tx_extra);
+  crypto::public_key get_tx_pub_key_from_extra(const transaction_prefix& tx);
   crypto::public_key get_tx_pub_key_from_extra(const transaction& tx);
   bool add_tx_pub_key_to_extra(transaction& tx, const crypto::public_key& tx_pub_key);
   bool add_extra_nonce_to_tx_extra(std::vector<uint8_t>& tx_extra, const blobdata& extra_nonce);
@@ -106,6 +112,7 @@ namespace cryptonote
   crypto::hash get_transaction_hash(const transaction& t);
   bool get_transaction_hash(const transaction& t, crypto::hash& res);
   bool get_transaction_hash(const transaction& t, crypto::hash& res, size_t& blob_size);
+  bool get_transaction_hash(const transaction& t, crypto::hash& res, size_t* blob_size);
   blobdata get_block_hashing_blob(const block& b);
   bool get_block_hash(const block& b, crypto::hash& res);
   crypto::hash get_block_hash(const block& b);

src/cryptonote_core/tx_pool.cpp

@@ -37,11 +37,7 @@
 #include "cryptonote_format_utils.h"
 #include "cryptonote_boost_serialization.h"
 #include "cryptonote_config.h"
-#if BLOCKCHAIN_DB == DB_LMDB
 #include "blockchain.h"
-#else
-#include "blockchain_storage.h"
-#endif
 #include "common/boost_serialization_helper.h"
 #include "common/int-util.h"
 #include "misc_language.h"
@@ -74,21 +70,22 @@ namespace cryptonote
     }
   }
   //---------------------------------------------------------------------------------
-#if BLOCKCHAIN_DB == DB_LMDB
   //---------------------------------------------------------------------------------
   tx_memory_pool::tx_memory_pool(Blockchain& bchs): m_blockchain(bchs)
   {
 
   }
-#else
-  tx_memory_pool::tx_memory_pool(blockchain_storage& bchs): m_blockchain(bchs)
-  {
-
-  }
-#endif
   //---------------------------------------------------------------------------------
   bool tx_memory_pool::add_tx(const transaction &tx, /*const crypto::hash& tx_prefix_hash,*/ const crypto::hash &id, size_t blob_size, tx_verification_context& tvc, bool kept_by_block, bool relayed, uint8_t version)
   {
+    if (tx.version == 0)
+    {
+      // v0 never accepted
+      LOG_PRINT_L1("transaction version 0 is invalid");
+      tvc.m_verifivation_failed = true;
+      return false;
+    }
+
     // we do not accept transactions that timed out before, unless they're
     // kept_by_block
     if (!kept_by_block && m_timed_out_transactions.find(id) != m_timed_out_transactions.end())
@@ -106,6 +103,11 @@ namespace cryptonote
       return false;
     }
 
+    // fee per kilobyte, size rounded up.
+    uint64_t fee;
+
+    if (tx.version == 1)
+    {
       uint64_t inputs_amount = 0;
       if(!get_inputs_money_amount(tx, inputs_amount))
       {
@@ -114,7 +116,6 @@ namespace cryptonote
       }
 
       uint64_t outputs_amount = get_outs_money_amount(tx);
-
       if(outputs_amount >= inputs_amount)
       {
         LOG_PRINT_L1("transaction use more money then it has: use " << print_money(outputs_amount) << ", have " << print_money(inputs_amount));
@@ -123,8 +124,13 @@ namespace cryptonote
         return false;
       }
 
-    // fee per kilobyte, size rounded up.
+      fee = inputs_amount - outputs_amount;
-    uint64_t fee = inputs_amount - outputs_amount;
+    }
+    else
+    {
+      fee = tx.rct_signatures.txnFee;
+    }
+
     uint64_t needed_fee = blob_size / 1024;
     needed_fee += (blob_size % 1024) ? 1 : 0;
     needed_fee *= FEE_PER_KB;
@@ -161,7 +167,7 @@ namespace cryptonote
 
     if (!m_blockchain.check_tx_outputs(tx, tvc))
     {
-      LOG_PRINT_L1("Transaction with id= "<< id << " has at least one invalid outout");
+      LOG_PRINT_L1("Transaction with id= "<< id << " has at least one invalid output");
       tvc.m_verifivation_failed = true;
       tvc.m_invalid_output = true;
       return false;
@@ -169,11 +175,9 @@ namespace cryptonote
 
     crypto::hash max_used_block_id = null_hash;
     uint64_t max_used_block_height = 0;
-#if BLOCKCHAIN_DB == DB_LMDB
+    tx_details txd;
-    bool ch_inp_res = m_blockchain.check_tx_inputs(tx, max_used_block_height, max_used_block_id, tvc, kept_by_block);
+    txd.tx = tx;
-#else
+    bool ch_inp_res = m_blockchain.check_tx_inputs(txd.tx, max_used_block_height, max_used_block_id, tvc, kept_by_block);
-    bool ch_inp_res = m_blockchain.check_tx_inputs(tx, max_used_block_height, max_used_block_id);
-#endif
     CRITICAL_REGION_LOCAL(m_transactions_lock);
     if(!ch_inp_res)
     {
@@ -181,11 +185,10 @@ namespace cryptonote
       // may become valid again, so ignore the failed inputs check.
       if(kept_by_block)
       {
-        auto txd_p = m_transactions.insert(transactions_container::value_type(id, tx_details()));
+        auto txd_p = m_transactions.insert(transactions_container::value_type(id, txd));
         CHECK_AND_ASSERT_MES(txd_p.second, false, "transaction already exists at inserting in memory pool");
         txd_p.first->second.blob_size = blob_size;
-        txd_p.first->second.tx = tx;
+        txd_p.first->second.fee = fee;
-        txd_p.first->second.fee = inputs_amount - outputs_amount;
         txd_p.first->second.max_used_block_id = null_hash;
         txd_p.first->second.max_used_block_height = 0;
         txd_p.first->second.kept_by_block = kept_by_block;
@@ -203,12 +206,11 @@ namespace cryptonote
     }else
     {
       //update transactions container
-      auto txd_p = m_transactions.insert(transactions_container::value_type(id, tx_details()));
+      auto txd_p = m_transactions.insert(transactions_container::value_type(id, txd));
       CHECK_AND_ASSERT_MES(txd_p.second, false, "intrnal error: transaction already exists at inserting in memorypool");
       txd_p.first->second.blob_size = blob_size;
-      txd_p.first->second.tx = tx;
       txd_p.first->second.kept_by_block = kept_by_block;
-      txd_p.first->second.fee = inputs_amount - outputs_amount;
+      txd_p.first->second.fee = fee;
       txd_p.first->second.max_used_block_id = max_used_block_id;
       txd_p.first->second.max_used_block_height = max_used_block_height;
       txd_p.first->second.last_failed_height = 0;
@@ -518,12 +520,8 @@ namespace cryptonote
         if(txd.last_failed_id == m_blockchain.get_block_id_by_height(txd.last_failed_height))
           return false;
         //check ring signature again, it is possible (with very small chance) that this transaction become again valid
-#if BLOCKCHAIN_DB == DB_LMDB
         tx_verification_context tvc;
         if(!m_blockchain.check_tx_inputs(txd.tx, txd.max_used_block_height, txd.max_used_block_id, tvc))
-#else
-        if(!m_blockchain.check_tx_inputs(txd.tx, txd.max_used_block_height, txd.max_used_block_id))
-#endif
         {
           txd.last_failed_height = m_blockchain.get_current_blockchain_height()-1;
           txd.last_failed_id = m_blockchain.get_block_id_by_height(txd.last_failed_height);
@@ -649,6 +647,7 @@ namespace cryptonote
   //---------------------------------------------------------------------------------
   size_t tx_memory_pool::validate(uint8_t version)
   {
+    CRITICAL_REGION_LOCAL(m_transactions_lock);
     size_t n_removed = 0;
     size_t tx_size_limit = (version < 2 ? TRANSACTION_SIZE_LIMIT_V1 : TRANSACTION_SIZE_LIMIT_V2);
     for (auto it = m_transactions.begin(); it != m_transactions.end(); ) {

src/cryptonote_core/tx_pool.h

@@ -48,11 +48,7 @@
 
 namespace cryptonote
 {
-#if BLOCKCHAIN_DB == DB_LMDB
   class Blockchain;
-#else
-  class blockchain_storage;
-#endif
   /************************************************************************/
   /*                                                                      */
   /************************************************************************/
@@ -93,16 +89,12 @@ namespace cryptonote
   class tx_memory_pool: boost::noncopyable
   {
   public:
-#if BLOCKCHAIN_DB == DB_LMDB
     /**
      * @brief Constructor
      *
      * @param bchs a Blockchain class instance, for getting chain info
      */
     tx_memory_pool(Blockchain& bchs);
-#else
-    tx_memory_pool(blockchain_storage& bchs);
-#endif
 
 
     /**
@@ -492,18 +484,7 @@ namespace cryptonote
     std::unordered_set<crypto::hash> m_timed_out_transactions;
 
     std::string m_config_folder;  //!< the folder to save state to
-#if BLOCKCHAIN_DB == DB_LMDB
     Blockchain& m_blockchain;  //!< reference to the Blockchain object
-#else
-    blockchain_storage& m_blockchain;
-#endif
-
-#if BLOCKCHAIN_DB == DB_LMDB
-#else
-#if defined(DEBUG_CREATE_BLOCK_TEMPLATE)
-    friend class blockchain_storage;
-#endif
-#endif
   };
 }
 

src/cryptonote_core/verification_context.h

@@ -47,6 +47,7 @@ namespace cryptonote
     bool m_too_big;
     bool m_overspend;
     bool m_fee_too_low;
+    bool m_not_rct;
   };
 
   struct block_verification_context

src/ringct/CMakeLists.txt

@@ -0,0 +1,59 @@
+# Copyright (c) 2016, The Monero Project
+#
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are
+# permitted provided that the following conditions are met:
+#
+# 1. Redistributions of source code must retain the above copyright notice, this list of
+#    conditions and the following disclaimer.
+#
+# 2. Redistributions in binary form must reproduce the above copyright notice, this list
+#    of conditions and the following disclaimer in the documentation and/or other
+#    materials provided with the distribution.
+#
+# 3. Neither the name of the copyright holder nor the names of its contributors may be
+#    used to endorse or promote products derived from this software without specific
+#    prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+# THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+# THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+set(ringct_sources
+  rctOps.cpp
+  rctSigs.cpp
+  rctTypes.cpp
+  rctCryptoOps.c)
+
+set(ringct_headers)
+
+set(ringct_private_headers
+  rctOps.h
+  rctSigs.h
+  rctTypes.h)
+
+bitmonero_private_headers(ringct
+  ${crypto_private_headers})
+bitmonero_add_library(ringct
+  ${ringct_sources}
+  ${ringct_headers}
+  ${ringct_private_headers})
+target_link_libraries(ringct
+  LINK_PUBLIC
+    common
+    crypto
+    ${Boost_DATE_TIME_LIBRARY}
+    ${Boost_PROGRAM_OPTIONS_LIBRARY}
+    ${Boost_SERIALIZATION_LIBRARY}
+  LINK_PRIVATE
+    ${Boost_FILESYSTEM_LIBRARY}
+    ${Boost_SYSTEM_LIBRARY}
+    ${Boost_THREAD_LIBRARY}
+    ${EXTRA_LIBRARIES})

src/ringct/rctCryptoOps.c

@@ -0,0 +1,221 @@
+// Copyright (c) 2014-2016, The Monero Project
+// 
+// All rights reserved.
+// 
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+// 
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+// 
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+// 
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+// 
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+// 
+// Parts of this file are originally copyright (c) 2012-2013 The Cryptonote developers
+
+#include <assert.h>
+#include <stdint.h>
+
+#include "crypto/crypto-ops.h"
+
+//DISABLE_VS_WARNINGS(4146 4244)
+
+void sc_reduce32copy(unsigned char * scopy, const unsigned char *s) {
+    int64_t s0 = 2097151 & load_3(s);
+    int64_t s1 = 2097151 & (load_4(s + 2) >> 5);
+    int64_t s2 = 2097151 & (load_3(s + 5) >> 2);
+    int64_t s3 = 2097151 & (load_4(s + 7) >> 7);
+    int64_t s4 = 2097151 & (load_4(s + 10) >> 4);
+    int64_t s5 = 2097151 & (load_3(s + 13) >> 1);
+    int64_t s6 = 2097151 & (load_4(s + 15) >> 6);
+    int64_t s7 = 2097151 & (load_3(s + 18) >> 3);
+    int64_t s8 = 2097151 & load_3(s + 21);
+    int64_t s9 = 2097151 & (load_4(s + 23) >> 5);
+    int64_t s10 = 2097151 & (load_3(s + 26) >> 2);
+    int64_t s11 = (load_4(s + 28) >> 7);
+    int64_t s12 = 0;
+    int64_t carry0;
+    int64_t carry1;
+    int64_t carry2;
+    int64_t carry3;
+    int64_t carry4;
+    int64_t carry5;
+    int64_t carry6;
+    int64_t carry7;
+    int64_t carry8;
+    int64_t carry9;
+    int64_t carry10;
+    int64_t carry11;
+
+    carry0 = (s0 + (1<<20)) >> 21;
+    s1 += carry0;
+    s0 -= carry0 << 21;
+    carry2 = (s2 + (1<<20)) >> 21;
+    s3 += carry2;
+    s2 -= carry2 << 21;
+    carry4 = (s4 + (1<<20)) >> 21;
+    s5 += carry4;
+    s4 -= carry4 << 21;
+    carry6 = (s6 + (1<<20)) >> 21;
+    s7 += carry6;
+    s6 -= carry6 << 21;
+    carry8 = (s8 + (1<<20)) >> 21;
+    s9 += carry8;
+    s8 -= carry8 << 21;
+    carry10 = (s10 + (1<<20)) >> 21;
+    s11 += carry10;
+    s10 -= carry10 << 21;
+
+    carry1 = (s1 + (1<<20)) >> 21;
+    s2 += carry1;
+    s1 -= carry1 << 21;
+    carry3 = (s3 + (1<<20)) >> 21;
+    s4 += carry3;
+    s3 -= carry3 << 21;
+    carry5 = (s5 + (1<<20)) >> 21;
+    s6 += carry5;
+    s5 -= carry5 << 21;
+    carry7 = (s7 + (1<<20)) >> 21;
+    s8 += carry7;
+    s7 -= carry7 << 21;
+    carry9 = (s9 + (1<<20)) >> 21;
+    s10 += carry9;
+    s9 -= carry9 << 21;
+    carry11 = (s11 + (1<<20)) >> 21;
+    s12 += carry11;
+    s11 -= carry11 << 21;
+
+    s0 += s12 * 666643;
+    s1 += s12 * 470296;
+    s2 += s12 * 654183;
+    s3 -= s12 * 997805;
+    s4 += s12 * 136657;
+    s5 -= s12 * 683901;
+    s12 = 0;
+
+    carry0 = s0 >> 21;
+    s1 += carry0;
+    s0 -= carry0 << 21;
+    carry1 = s1 >> 21;
+    s2 += carry1;
+    s1 -= carry1 << 21;
+    carry2 = s2 >> 21;
+    s3 += carry2;
+    s2 -= carry2 << 21;
+    carry3 = s3 >> 21;
+    s4 += carry3;
+    s3 -= carry3 << 21;
+    carry4 = s4 >> 21;
+    s5 += carry4;
+    s4 -= carry4 << 21;
+    carry5 = s5 >> 21;
+    s6 += carry5;
+    s5 -= carry5 << 21;
+    carry6 = s6 >> 21;
+    s7 += carry6;
+    s6 -= carry6 << 21;
+    carry7 = s7 >> 21;
+    s8 += carry7;
+    s7 -= carry7 << 21;
+    carry8 = s8 >> 21;
+    s9 += carry8;
+    s8 -= carry8 << 21;
+    carry9 = s9 >> 21;
+    s10 += carry9;
+    s9 -= carry9 << 21;
+    carry10 = s10 >> 21;
+    s11 += carry10;
+    s10 -= carry10 << 21;
+    carry11 = s11 >> 21;
+    s12 += carry11;
+    s11 -= carry11 << 21;
+
+    s0 += s12 * 666643;
+    s1 += s12 * 470296;
+    s2 += s12 * 654183;
+    s3 -= s12 * 997805;
+    s4 += s12 * 136657;
+    s5 -= s12 * 683901;
+
+    carry0 = s0 >> 21;
+    s1 += carry0;
+    s0 -= carry0 << 21;
+    carry1 = s1 >> 21;
+    s2 += carry1;
+    s1 -= carry1 << 21;
+    carry2 = s2 >> 21;
+    s3 += carry2;
+    s2 -= carry2 << 21;
+    carry3 = s3 >> 21;
+    s4 += carry3;
+    s3 -= carry3 << 21;
+    carry4 = s4 >> 21;
+    s5 += carry4;
+    s4 -= carry4 << 21;
+    carry5 = s5 >> 21;
+    s6 += carry5;
+    s5 -= carry5 << 21;
+    carry6 = s6 >> 21;
+    s7 += carry6;
+    s6 -= carry6 << 21;
+    carry7 = s7 >> 21;
+    s8 += carry7;
+    s7 -= carry7 << 21;
+    carry8 = s8 >> 21;
+    s9 += carry8;
+    s8 -= carry8 << 21;
+    carry9 = s9 >> 21;
+    s10 += carry9;
+    s9 -= carry9 << 21;
+    carry10 = s10 >> 21;
+    s11 += carry10;
+    s10 -= carry10 << 21;
+
+    scopy[0] = s0 >> 0;
+    scopy[1] = s0 >> 8;
+    scopy[2] = (s0 >> 16) | (s1 << 5);
+    scopy[3] = s1 >> 3;
+    scopy[4] = s1 >> 11;
+    scopy[5] = (s1 >> 19) | (s2 << 2);
+    scopy[6] = s2 >> 6;
+    scopy[7] = (s2 >> 14) | (s3 << 7);
+    scopy[8] = s3 >> 1;
+    scopy[9] = s3 >> 9;
+    scopy[10] = (s3 >> 17) | (s4 << 4);
+    scopy[11] = s4 >> 4;
+    scopy[12] = s4 >> 12;
+    scopy[13] = (s4 >> 20) | (s5 << 1);
+    scopy[14] = s5 >> 7;
+    scopy[15] = (s5 >> 15) | (s6 << 6);
+    scopy[16] = s6 >> 2;
+    scopy[17] = s6 >> 10;
+    scopy[18] = (s6 >> 18) | (s7 << 3);
+    scopy[19] = s7 >> 5;
+    scopy[20] = s7 >> 13;
+    scopy[21] = s8 >> 0;
+    scopy[22] = s8 >> 8;
+    scopy[23] = (s8 >> 16) | (s9 << 5);
+    scopy[24] = s9 >> 3;
+    scopy[25] = s9 >> 11;
+    scopy[26] = (s9 >> 19) | (s10 << 2);
+    scopy[27] = s10 >> 6;
+    scopy[28] = (s10 >> 14) | (s11 << 7);
+    scopy[29] = s11 >> 1;
+    scopy[30] = s11 >> 9;
+    scopy[31] = s11 >> 17;
+}

src/ringct/rctCryptoOps.h

@@ -0,0 +1,37 @@
+// Copyright (c) 2014-2016, The Monero Project
+// 
+// All rights reserved.
+// 
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+// 
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+// 
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+// 
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+// 
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+// 
+// Parts of this file are originally copyright (c) 2012-2013 The Cryptonote developers
+
+#pragma once
+
+extern "C" {
+#include "crypto/crypto-ops.h"
+}
+
+void sc_reduce32copy(unsigned char * scopy, const unsigned char *s);

src/ringct/rctOps.cpp

@@ -0,0 +1,475 @@
+// Copyright (c) 2016, Monero Research Labs
+//
+// Author: Shen Noether <shen.noether@gmx.com>
+//
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+//
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+//
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+//
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+#include "misc_log_ex.h"
+#include "rctOps.h"
+using namespace crypto;
+using namespace std;
+
+namespace rct {
+
+    //Various key initialization functions
+
+    //Creates a zero scalar
+    void zero(key &zero) {
+        memset(&zero, 0, 32);
+    }
+
+    //Creates a zero scalar
+    key zero() {
+        static const key z = { {0x00, 0x00, 0x00,0x00 , 0x00, 0x00, 0x00,0x00 , 0x00, 0x00, 0x00,0x00 , 0x00, 0x00, 0x00,0x00 , 0x00, 0x00, 0x00,0x00 , 0x00, 0x00, 0x00,0x00 , 0x00, 0x00, 0x00,0x00 , 0x00, 0x00, 0x00,0x00  } };
+        return z;
+    }
+
+    //Creates a zero elliptic curve point
+    void identity(key &Id) {
+        Id[0] = (unsigned char)(0x01);
+        memset(Id.bytes+1, 0, 31);
+    }
+
+    //Creates a zero elliptic curve point
+    key identity() {
+        key Id;
+        Id[0] = (unsigned char)(0x01);
+        memset(Id.bytes+1, 0, 31);
+        return Id;
+    }
+
+    //copies a scalar or point
+    void copy(key &AA, const key &A) {
+        memcpy(&AA, &A, 32);
+    }
+
+    //copies a scalar or point
+    key copy(const key &A) {
+        key AA;
+        memcpy(&AA, &A, 32);
+        return AA;
+    }
+
+
+    //initializes a key matrix;
+    //first parameter is rows,
+    //second is columns
+    keyM keyMInit(int rows, int cols) {
+        keyM rv(cols);
+        int i = 0;
+        for (i = 0 ; i < cols ; i++) {
+            rv[i] = keyV(rows);
+        }
+        return rv;
+    }
+
+
+
+
+    //Various key generation functions
+
+    //generates a random scalar which can be used as a secret key or mask
+    void skGen(key &sk) {
+        sk = crypto::rand<key>();
+        sc_reduce32(sk.bytes);
+    }
+
+    //generates a random scalar which can be used as a secret key or mask
+    key skGen() {
+        key sk = crypto::rand<key>();
+        sc_reduce32(sk.bytes);
+        return sk;
+    }
+
+    //Generates a vector of secret key
+    //Mainly used in testing
+    keyV skvGen(int rows ) {
+        keyV rv(rows);
+        int i = 0;
+        for (i = 0 ; i < rows ; i++) {
+            skGen(rv[i]);
+        }
+        return rv;
+    }
+
+    //generates a random curve point (for testing)
+    key  pkGen() {
+        key sk = skGen();
+        key pk = scalarmultBase(sk);
+        return pk;
+    }
+
+    //generates a random secret and corresponding public key
+    void skpkGen(key &sk, key &pk) {
+        skGen(sk);
+        scalarmultBase(pk, sk);
+    }
+
+    //generates a random secret and corresponding public key
+    tuple<key, key>  skpkGen() {
+        key sk = skGen();
+        key pk = scalarmultBase(sk);
+        return make_tuple(sk, pk);
+    }
+
+    //generates C =aG + bH from b, a is given..
+    void genC(key & C, const key & a, xmr_amount amount) {
+        key bH = scalarmultH(d2h(amount));
+        addKeys1(C, a, bH);
+    }
+
+    //generates a <secret , public> / Pedersen commitment to the amount
+    tuple<ctkey, ctkey> ctskpkGen(xmr_amount amount) {
+        ctkey sk, pk;
+        skpkGen(sk.dest, pk.dest);
+        skpkGen(sk.mask, pk.mask);
+        key am = d2h(amount);
+        key bH = scalarmultH(am);
+        addKeys(pk.mask, pk.mask, bH);
+        return make_tuple(sk, pk);
+    }
+    
+    
+    //generates a <secret , public> / Pedersen commitment but takes bH as input 
+    tuple<ctkey, ctkey> ctskpkGen(key bH) {
+        ctkey sk, pk;
+        skpkGen(sk.dest, pk.dest);
+        skpkGen(sk.mask, pk.mask);
+        addKeys(pk.mask, pk.mask, bH);
+        return make_tuple(sk, pk);
+    }
+    
+    key zeroCommit(xmr_amount amount) {
+        key mask = identity();
+        mask = scalarmultBase(mask);
+        key am = d2h(amount);
+        key bH = scalarmultH(am);
+        addKeys(mask, mask, bH);
+        return mask;
+    }
+
+    key commit(xmr_amount amount, key mask) {
+        mask = scalarmultBase(mask);
+        key am = d2h(amount);
+        key bH = scalarmultH(am);
+        addKeys(mask, mask, bH);
+        return mask;
+    }
+
+    //generates a random uint long long (for testing)
+    xmr_amount randXmrAmount(xmr_amount upperlimit) {
+        return h2d(skGen()) % (upperlimit);
+    }
+
+    //Scalar multiplications of curve points
+
+    //does a * G where a is a scalar and G is the curve basepoint
+    void scalarmultBase(key &aG,const key &a) {
+        ge_p3 point;
+        sc_reduce32copy(aG.bytes, a.bytes); //do this beforehand!
+        ge_scalarmult_base(&point, aG.bytes);
+        ge_p3_tobytes(aG.bytes, &point);
+    }
+
+    //does a * G where a is a scalar and G is the curve basepoint
+    key scalarmultBase(const key & a) {
+        ge_p3 point;
+        key aG;
+        sc_reduce32copy(aG.bytes, a.bytes); //do this beforehand
+        ge_scalarmult_base(&point, aG.bytes);
+        ge_p3_tobytes(aG.bytes, &point);
+        return aG;
+    }
+
+    //does a * P where a is a scalar and P is an arbitrary point
+    void scalarmultKey(key & aP, const key &P, const key &a) {
+        ge_p3 A;
+        ge_p2 R;
+        CHECK_AND_ASSERT_THROW_MES(ge_frombytes_vartime(&A, P.bytes) == 0, "ge_frombytes_vartime failed at "+boost::lexical_cast<std::string>(__LINE__));
+        ge_scalarmult(&R, a.bytes, &A);
+        ge_tobytes(aP.bytes, &R);
+    }
+
+    //does a * P where a is a scalar and P is an arbitrary point
+    key scalarmultKey(const key & P, const key & a) {
+        ge_p3 A;
+        ge_p2 R;
+        CHECK_AND_ASSERT_THROW_MES(ge_frombytes_vartime(&A, P.bytes) == 0, "ge_frombytes_vartime failed at "+boost::lexical_cast<std::string>(__LINE__));
+        ge_scalarmult(&R, a.bytes, &A);
+        key aP;
+        ge_tobytes(aP.bytes, &R);
+        return aP;
+    }
+
+
+    //Computes aH where H= toPoint(cn_fast_hash(G)), G the basepoint
+    key scalarmultH(const key & a) {
+        ge_p3 A;
+        ge_p2 R;
+        CHECK_AND_ASSERT_THROW_MES(ge_frombytes_vartime(&A, H.bytes) == 0, "ge_frombytes_vartime failed at "+boost::lexical_cast<std::string>(__LINE__));
+        ge_scalarmult(&R, a.bytes, &A);
+        key aP;
+        ge_tobytes(aP.bytes, &R);
+        return aP;
+    }
+
+    //Curve addition / subtractions
+
+    //for curve points: AB = A + B
+    void addKeys(key &AB, const key &A, const key &B) {
+        ge_p3 B2, A2;
+        CHECK_AND_ASSERT_THROW_MES(ge_frombytes_vartime(&B2, B.bytes) == 0, "ge_frombytes_vartime failed at "+boost::lexical_cast<std::string>(__LINE__));
+        CHECK_AND_ASSERT_THROW_MES(ge_frombytes_vartime(&A2, A.bytes) == 0, "ge_frombytes_vartime failed at "+boost::lexical_cast<std::string>(__LINE__));
+        ge_cached tmp2;
+        ge_p3_to_cached(&tmp2, &B2);
+        ge_p1p1 tmp3;
+        ge_add(&tmp3, &A2, &tmp2);
+        ge_p1p1_to_p3(&A2, &tmp3);
+        ge_p3_tobytes(AB.bytes, &A2);
+    }
+
+
+    //addKeys1
+    //aGB = aG + B where a is a scalar, G is the basepoint, and B is a point
+    void addKeys1(key &aGB, const key &a, const key & B) {
+        key aG = scalarmultBase(a);
+        addKeys(aGB, aG, B);
+    }
+
+    //addKeys2
+    //aGbB = aG + bB where a, b are scalars, G is the basepoint and B is a point
+    void addKeys2(key &aGbB, const key &a, const key &b, const key & B) {
+        ge_p2 rv;
+        ge_p3 B2;
+        CHECK_AND_ASSERT_THROW_MES(ge_frombytes_vartime(&B2, B.bytes) == 0, "ge_frombytes_vartime failed at "+boost::lexical_cast<std::string>(__LINE__));
+        ge_double_scalarmult_base_vartime(&rv, b.bytes, &B2, a.bytes);
+        ge_tobytes(aGbB.bytes, &rv);
+    }
+
+    //Does some precomputation to make addKeys3 more efficient
+    // input B a curve point and output a ge_dsmp which has precomputation applied
+    void precomp(ge_dsmp rv, const key & B) {
+        ge_p3 B2;
+        CHECK_AND_ASSERT_THROW_MES(ge_frombytes_vartime(&B2, B.bytes) == 0, "ge_frombytes_vartime failed at "+boost::lexical_cast<std::string>(__LINE__));
+        ge_dsm_precomp(rv, &B2);
+    }
+
+    //addKeys3
+    //aAbB = a*A + b*B where a, b are scalars, A, B are curve points
+    //B must be input after applying "precomp"
+    void addKeys3(key &aAbB, const key &a, const key &A, const key &b, const ge_dsmp B) {
+        ge_p2 rv;
+        ge_p3 A2;
+        CHECK_AND_ASSERT_THROW_MES(ge_frombytes_vartime(&A2, A.bytes) == 0, "ge_frombytes_vartime failed at "+boost::lexical_cast<std::string>(__LINE__));
+        ge_double_scalarmult_precomp_vartime(&rv, a.bytes, &A2, b.bytes, B);
+        ge_tobytes(aAbB.bytes, &rv);
+    }
+
+
+    //subtract Keys (subtracts curve points)
+    //AB = A - B where A, B are curve points
+    void subKeys(key & AB, const key &A, const key &B) {
+        ge_p3 B2, A2;
+        CHECK_AND_ASSERT_THROW_MES(ge_frombytes_vartime(&B2, B.bytes) == 0, "ge_frombytes_vartime failed at "+boost::lexical_cast<std::string>(__LINE__));
+        CHECK_AND_ASSERT_THROW_MES(ge_frombytes_vartime(&A2, A.bytes) == 0, "ge_frombytes_vartime failed at "+boost::lexical_cast<std::string>(__LINE__));
+        ge_cached tmp2;
+        ge_p3_to_cached(&tmp2, &B2);
+        ge_p1p1 tmp3;
+        ge_sub(&tmp3, &A2, &tmp2);
+        ge_p1p1_to_p3(&A2, &tmp3);
+        ge_p3_tobytes(AB.bytes, &A2);
+    }
+
+    //checks if A, B are equal as curve points
+    //without doing curve operations
+    bool equalKeys(const key & a, const key & b) {
+        bool rv = true;
+        for (int i = 0; i < 32; ++i) {
+          if (a.bytes[i] != b.bytes[i]) {
+            rv = false;
+          }
+        }
+        return rv;
+    }
+
+    //Hashing - cn_fast_hash
+    //be careful these are also in crypto namespace
+    //cn_fast_hash for arbitrary multiples of 32 bytes
+    void cn_fast_hash(key &hash, const void * data, const std::size_t l) {
+        keccak((uint8_t *)data, l, hash.bytes, 32);
+    }
+    
+    void hash_to_scalar(key &hash, const void * data, const std::size_t l) {
+        cn_fast_hash(hash, data, l);
+        sc_reduce32(hash.bytes);
+    }
+
+    //cn_fast_hash for a 32 byte key
+    void cn_fast_hash(key & hash, const key & in) {
+        keccak((uint8_t *)in.bytes, 32, hash.bytes, 32);
+    }
+    
+    void hash_to_scalar(key & hash, const key & in) {
+        cn_fast_hash(hash, in);
+        sc_reduce32(hash.bytes);
+    }
+
+    //cn_fast_hash for a 32 byte key
+    key cn_fast_hash(const key & in) {
+        key hash;
+        keccak((uint8_t *)in.bytes, 32, hash.bytes, 32);
+        return hash;
+    }
+    
+     key hash_to_scalar(const key & in) {
+        key hash = cn_fast_hash(in);
+        sc_reduce32(hash.bytes);
+        return hash;
+     }
+    
+    //cn_fast_hash for a 128 byte unsigned char
+    key cn_fast_hash128(const void * in) {
+        key hash;
+        keccak((uint8_t *)in, 128, hash.bytes, 32);
+        return hash;
+    }
+    
+    key hash_to_scalar128(const void * in) {
+        key hash = cn_fast_hash128(in);
+        sc_reduce32(hash.bytes);
+        return hash;
+    }
+    
+    //cn_fast_hash for multisig purpose
+    //This takes the outputs and commitments
+    //and hashes them into a 32 byte sized key
+    key cn_fast_hash(ctkeyV PC) {
+        key rv = identity();
+        std::size_t l = (std::size_t)PC.size();
+        size_t i = 0, j = 0;
+        vector<char> m(l * 64);
+        for (i = 0 ; i < l ; i++) {
+            memcpy(&m[i * 64], &PC[i].dest, 32);
+            memcpy(&m[i * 64 + 32], &PC[i].mask, 32);
+        }
+        cn_fast_hash(rv, &m[0], 64*l);
+        return rv;
+    }
+    
+    key hash_to_scalar(ctkeyV PC) {
+        key rv = cn_fast_hash(PC);
+        sc_reduce32(rv.bytes);
+        return rv;
+    }
+    
+   //cn_fast_hash for a key-vector of arbitrary length
+   //this is useful since you take a number of keys
+   //put them in the key vector and it concatenates them
+   //and then hashes them
+   key cn_fast_hash(const keyV &keys) {
+       size_t l = keys.size();
+       vector<unsigned char> m(l * 32);
+       size_t i;
+       for (i = 0 ; i < l ; i++) {
+           memcpy(&m[i * 32], keys[i].bytes, 32);
+       }
+       key rv;
+       cn_fast_hash(rv, &m[0], 32 * l);
+       //dp(rv);
+       return rv;
+   }
+   
+   key hash_to_scalar(const keyV &keys) {
+       key rv = cn_fast_hash(keys);
+       sc_reduce32(rv.bytes);
+       return rv;
+   }
+
+    key hashToPointSimple(const key & hh) {
+        key pointk;
+        ge_p1p1 point2;
+        ge_p2 point;
+        ge_p3 res;
+        key h = cn_fast_hash(hh); 
+        CHECK_AND_ASSERT_THROW_MES(ge_frombytes_vartime(&res, h.bytes) == 0, "ge_frombytes_vartime failed at "+boost::lexical_cast<std::string>(__LINE__));
+        ge_p3_to_p2(&point, &res);
+        ge_mul8(&point2, &point);
+        ge_p1p1_to_p3(&res, &point2);
+        ge_p3_tobytes(pointk.bytes, &res);
+        return pointk;
+    }    
+    
+    key hashToPoint(const key & hh) {
+        key pointk;
+        ge_p2 point;
+        ge_p1p1 point2;
+        ge_p3 res;
+        key h = cn_fast_hash(hh); 
+        ge_fromfe_frombytes_vartime(&point, h.bytes);
+        ge_mul8(&point2, &point);
+        ge_p1p1_to_p3(&res, &point2);        
+        ge_p3_tobytes(pointk.bytes, &res);
+        return pointk;
+    }
+
+    void hashToPoint(key & pointk, const key & hh) {
+        ge_p2 point;
+        ge_p1p1 point2;
+        ge_p3 res;
+        key h = cn_fast_hash(hh); 
+        ge_fromfe_frombytes_vartime(&point, h.bytes);
+        ge_mul8(&point2, &point);
+        ge_p1p1_to_p3(&res, &point2);        
+        ge_p3_tobytes(pointk.bytes, &res);
+    }    
+
+    //sums a vector of curve points (for scalars use sc_add)
+    void sumKeys(key & Csum, const keyV &  Cis) {
+        identity(Csum);
+        size_t i = 0;
+        for (i = 0; i < Cis.size(); i++) {
+            addKeys(Csum, Csum, Cis[i]);
+        }
+    }
+
+    //Elliptic Curve Diffie Helman: encodes and decodes the amount b and mask a
+    // where C= aG + bH
+    void ecdhEncode(ecdhTuple & unmasked, const key & sharedSec) {
+        key sharedSec1 = hash_to_scalar(sharedSec);
+        key sharedSec2 = hash_to_scalar(sharedSec1);
+        //encode
+        sc_add(unmasked.mask.bytes, unmasked.mask.bytes, sharedSec1.bytes);
+        sc_add(unmasked.amount.bytes, unmasked.amount.bytes, sharedSec2.bytes);
+    }
+    void ecdhDecode(ecdhTuple & masked, const key & sharedSec) {
+        key sharedSec1 = hash_to_scalar(sharedSec);
+        key sharedSec2 = hash_to_scalar(sharedSec1);
+        //decode
+        sc_sub(masked.mask.bytes, masked.mask.bytes, sharedSec1.bytes);
+        sc_sub(masked.amount.bytes, masked.amount.bytes, sharedSec2.bytes);
+    }
+}

src/ringct/rctOps.h

@@ -0,0 +1,171 @@
+//#define DBG
+// Copyright (c) 2016, Monero Research Labs
+//
+// Author: Shen Noether <shen.noether@gmx.com>
+//
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+//
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+//
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+//
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+#pragma once
+
+#ifndef RCTOPS_H
+#define RCTOPS_H
+
+#include <cstddef>
+#include <mutex>
+#include <vector>
+#include <tuple>
+
+#include "crypto/generic-ops.h"
+
+extern "C" {
+#include "crypto/random.h"
+#include "crypto/keccak.h"
+#include "rctCryptoOps.h"
+}
+#include "crypto/crypto.h"
+
+#include "rctTypes.h"
+
+//Define this flag when debugging to get additional info on the console
+#ifdef DBG
+#define DP(x) dp(x)
+#else
+#define DP(x)
+#endif
+
+using namespace std;
+using namespace crypto;
+
+namespace rct {
+
+    //Various key initialization functions
+
+    //Creates a zero scalar
+    key zero();
+    void zero(key &z);
+    //Creates a zero elliptic curve point
+    key identity();
+    void identity(key &Id);
+    //copies a scalar or point
+    void copy(key &AA, const key &A);
+    key copy(const key & AA);
+    //initializes a key matrix;
+    //first parameter is rows, 
+    //second is columns
+    keyM keyMInit(int, int);
+
+    //Various key generation functions        
+
+    //generates a random scalar which can be used as a secret key or mask
+    key skGen();
+    void skGen(key &);
+    
+    //generates a vector of secret keys of size "int"
+    keyV skvGen(int );
+    
+    //generates a random curve point (for testing)
+    key pkGen();
+    //generates a random secret and corresponding public key
+    void skpkGen(key &sk, key &pk);
+    tuple<key, key> skpkGen();
+    //generates a <secret , public> / Pedersen commitment to the amount
+    tuple<ctkey, ctkey> ctskpkGen(xmr_amount amount);
+    //generates C =aG + bH from b, a is random
+    void genC(key & C, const key & a, xmr_amount amount);
+    //this one is mainly for testing, can take arbitrary amounts..
+    tuple<ctkey, ctkey> ctskpkGen(key bH);
+    // make a pedersen commitment with given key
+    key commit(xmr_amount amount, key mask);
+    // make a pedersen commitment with zero key
+    key zeroCommit(xmr_amount amount);
+    //generates a random uint long long
+    xmr_amount randXmrAmount(xmr_amount upperlimit);
+
+    //Scalar multiplications of curve points        
+
+    //does a * G where a is a scalar and G is the curve basepoint
+    void scalarmultBase(key & aG, const key &a);
+    key scalarmultBase(const key & a);
+    //does a * P where a is a scalar and P is an arbitrary point
+    void scalarmultKey(key &aP, const key &P, const key &a);
+    key scalarmultKey(const key &P, const key &a);
+    //Computes aH where H= toPoint(cn_fast_hash(G)), G the basepoint
+    key scalarmultH(const key & a);
+
+    //Curve addition / subtractions
+
+    //for curve points: AB = A + B
+    void addKeys(key &AB, const key &A, const key &B);
+    //aGB = aG + B where a is a scalar, G is the basepoint, and B is a point
+    void addKeys1(key &aGB, const key &a, const key & B);
+    //aGbB = aG + bB where a, b are scalars, G is the basepoint and B is a point
+    void addKeys2(key &aGbB, const key &a, const key &b, const key &B);
+    //Does some precomputation to make addKeys3 more efficient
+    // input B a curve point and output a ge_dsmp which has precomputation applied
+    void precomp(ge_dsmp rv, const key &B);
+    //aAbB = a*A + b*B where a, b are scalars, A, B are curve points
+    //B must be input after applying "precomp"
+    void addKeys3(key &aAbB, const key &a, const key &A, const key &b, const ge_dsmp B);
+    //AB = A - B where A, B are curve points
+    void subKeys(key &AB, const key &A, const  key &B);
+    //checks if A, B are equal as curve points
+    bool equalKeys(const key & A, const key & B);
+
+    //Hashing - cn_fast_hash
+    //be careful these are also in crypto namespace
+    //cn_fast_hash for arbitrary l multiples of 32 bytes 
+    void cn_fast_hash(key &hash, const void * data, const size_t l);
+    void hash_to_scalar(key &hash, const void * data, const size_t l);
+    //cn_fast_hash for a 32 byte key
+    void cn_fast_hash(key &hash, const key &in);
+    void hash_to_scalar(key &hash, const key &in);
+    //cn_fast_hash for a 32 byte key
+    key cn_fast_hash(const key &in);
+    key hash_to_scalar(const key &in);
+    //for mg sigs
+    key cn_fast_hash128(const void * in);
+    key hash_to_scalar128(const void * in);
+    key cn_fast_hash(ctkeyV PC);
+    key hash_to_scalar(ctkeyV PC);
+    //for mg sigs 
+    key cn_fast_hash(const keyV &keys);
+    key hash_to_scalar(const keyV &keys);
+
+    //returns hashToPoint as described in https://github.com/ShenNoether/ge_fromfe_writeup 
+    key hashToPointSimple(const key &in);
+    key hashToPoint(const key &in);
+    void hashToPoint(key &out, const key &in);
+
+    //sums a vector of curve points (for scalars use sc_add)
+    void sumKeys(key & Csum, const key &Cis);
+
+    //Elliptic Curve Diffie Helman: encodes and decodes the amount b and mask a
+    // where C= aG + bH
+    void ecdhEncode(ecdhTuple & unmasked, const key & sharedSec);
+    void ecdhDecode(ecdhTuple & masked, const key & sharedSec);
+}
+#endif  /* RCTOPS_H */

src/ringct/rctSigs.cpp

@@ -0,0 +1,875 @@
+// Copyright (c) 2016, Monero Research Labs
+//
+// Author: Shen Noether <shen.noether@gmx.com>
+// 
+// All rights reserved.
+// 
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+// 
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+// 
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+// 
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+// 
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+#include "misc_log_ex.h"
+#include "rctSigs.h"
+#include "cryptonote_core/cryptonote_format_utils.h"
+
+using namespace crypto;
+using namespace std;
+
+namespace rct {
+    
+    //Schnorr Non-linkable
+    //Gen Gives a signature (L1, s1, s2) proving that the sender knows "x" such that xG = one of P1 or P2
+    //Ver Verifies that signer knows an "x" such that xG = one of P1 or P2
+    //These are called in the below ASNL sig generation    
+    
+    void GenSchnorrNonLinkable(key & L1, key & s1, key & s2, const key & x, const key & P1, const key & P2, int index) {
+        key c1, c2, L2;
+        key a = skGen();
+        if (index == 0) {
+            scalarmultBase(L1, a);
+            hash_to_scalar(c2, L1);
+            skGen(s2);
+            addKeys2(L2, s2, c2, P2);
+            hash_to_scalar(c1, L2);
+            //s1 = a - x * c1
+            sc_mulsub(s1.bytes, x.bytes, c1.bytes, a.bytes);
+        }
+        else if (index == 1) {
+            scalarmultBase(L2, a);
+            hash_to_scalar(c1, L2);
+            skGen(s1);
+            addKeys2(L1, s1, c1, P1);
+            hash_to_scalar(c2, L1);
+            sc_mulsub(s2.bytes, x.bytes, c2.bytes, a.bytes);
+        }
+        else {
+          throw std::runtime_error("GenSchnorrNonLinkable: invalid index (should be 0 or 1)");
+        }
+    }
+
+    //Schnorr Non-linkable
+    //Gen Gives a signature (L1, s1, s2) proving that the sender knows "x" such that xG = one of P1 or P2
+    //Ver Verifies that signer knows an "x" such that xG = one of P1 or P2
+    //These are called in the below ASNL sig generation        
+    bool VerSchnorrNonLinkable(const key & P1, const key & P2, const key & L1, const key & s1, const key & s2) {
+        key c2, L2, c1, L1p;
+        hash_to_scalar(c2, L1);
+        addKeys2(L2, s2, c2, P2);
+        hash_to_scalar(c1, L2);
+        addKeys2(L1p, s1, c1, P1);
+        
+        return equalKeys(L1, L1p);
+    }
+    
+    //Aggregate Schnorr Non-linkable Ring Signature (ASNL)
+    // c.f. http://eprint.iacr.org/2015/1098 section 5. 
+    // These are used in range proofs (alternatively Borromean could be used)
+    // Gen gives a signature which proves the signer knows, for each i, 
+    //   an x[i] such that x[i]G = one of P1[i] or P2[i]
+    // Ver Verifies the signer knows a key for one of P1[i], P2[i] at each i
+    asnlSig GenASNL(key64 x, key64 P1, key64 P2, bits indices) {
+        DP("Generating Aggregate Schnorr Non-linkable Ring Signature\n");
+        key64 s1;
+        int j = 0;
+        asnlSig rv;
+        rv.s = zero();
+        for (j = 0; j < ATOMS; j++) {
+            GenSchnorrNonLinkable(rv.L1[j], s1[j], rv.s2[j], x[j], P1[j], P2[j], (int)indices[j]);
+            sc_add(rv.s.bytes, rv.s.bytes, s1[j].bytes);
+        }
+        return rv;
+    }
+
+    //Aggregate Schnorr Non-linkable Ring Signature (ASNL)
+    // c.f. http://eprint.iacr.org/2015/1098 section 5. 
+    // These are used in range proofs (alternatively Borromean could be used)
+    // Gen gives a signature which proves the signer knows, for each i, 
+    //   an x[i] such that x[i]G = one of P1[i] or P2[i]
+    // Ver Verifies the signer knows a key for one of P1[i], P2[i] at each i    
+    bool VerASNL(const key64 P1, const key64 P2, const asnlSig &as) {
+        DP("Verifying Aggregate Schnorr Non-linkable Ring Signature\n");
+        key LHS = identity();
+        key RHS = scalarmultBase(as.s);
+        key c2, L2, c1;
+        int j = 0;
+        for (j = 0; j < ATOMS; j++) {
+            hash_to_scalar(c2, as.L1[j]);
+            addKeys2(L2, as.s2[j], c2, P2[j]);
+            addKeys(LHS, LHS, as.L1[j]);
+            hash_to_scalar(c1, L2);
+            addKeys(RHS, RHS, scalarmultKey(P1[j], c1));
+        }
+        key cc;
+        sc_sub(cc.bytes, LHS.bytes, RHS.bytes);
+        return sc_isnonzero(cc.bytes) == 0;
+    }
+    
+    //Multilayered Spontaneous Anonymous Group Signatures (MLSAG signatures)
+    //These are aka MG signatutes in earlier drafts of the ring ct paper
+    // c.f. http://eprint.iacr.org/2015/1098 section 2. 
+    // keyImageV just does I[i] = xx[i] * Hash(xx[i] * G) for each i
+    // Gen creates a signature which proves that for some column in the keymatrix "pk"
+    //   the signer knows a secret key for each row in that column
+    // Ver verifies that the MG sig was created correctly
+    keyV keyImageV(const keyV &xx) {
+        keyV II(xx.size());
+        size_t i = 0;
+        for (i = 0; i < xx.size(); i++) {
+            II[i] = scalarmultKey(hashToPoint(scalarmultBase(xx[i])), xx[i]);
+        }
+        return II;
+    }
+    
+    
+    //Multilayered Spontaneous Anonymous Group Signatures (MLSAG signatures)
+    //This is a just slghtly more efficient version than the ones described below
+    //(will be explained in more detail in Ring Multisig paper
+    //These are aka MG signatutes in earlier drafts of the ring ct paper
+    // c.f. http://eprint.iacr.org/2015/1098 section 2. 
+    // keyImageV just does I[i] = xx[i] * Hash(xx[i] * G) for each i
+    // Gen creates a signature which proves that for some column in the keymatrix "pk"
+    //   the signer knows a secret key for each row in that column
+    // Ver verifies that the MG sig was created correctly        
+    mgSig MLSAG_Gen(key message, const keyM & pk, const keyV & xx, const unsigned int index, size_t dsRows) {
+        mgSig rv;
+        size_t cols = pk.size();
+        CHECK_AND_ASSERT_THROW_MES(cols >= 2, "Error! What is c if cols = 1!");
+        CHECK_AND_ASSERT_THROW_MES(index < cols, "Index out of range");
+        size_t rows = pk[0].size();
+        CHECK_AND_ASSERT_THROW_MES(rows >= 1, "Empty pk");
+        for (size_t i = 1; i < cols; ++i) {
+          CHECK_AND_ASSERT_THROW_MES(pk[i].size() == rows, "pk is not rectangular");
+        }
+        CHECK_AND_ASSERT_THROW_MES(xx.size() == rows, "Bad xx size");
+        CHECK_AND_ASSERT_THROW_MES(dsRows <= rows, "Bad dsRows size");
+
+        size_t i = 0, j = 0, ii = 0;
+        key c, c_old, L, R, Hi;
+        sc_0(c_old.bytes);
+        vector<geDsmp> Ip(dsRows);
+        rv.II = keyV(dsRows);
+        keyV alpha(rows);
+        keyV aG(rows);
+        rv.ss = keyM(cols, aG);
+        keyV aHP(dsRows);
+        keyV toHash(1 + 3 * dsRows + 2 * (rows - dsRows));
+        toHash[0] = message;
+        DP("here1");
+        for (i = 0; i < dsRows; i++) {
+            skpkGen(alpha[i], aG[i]); //need to save alphas for later..
+            Hi = hashToPoint(pk[index][i]);
+            aHP[i] = scalarmultKey(Hi, alpha[i]);
+            toHash[3 * i + 1] = pk[index][i];
+            toHash[3 * i + 2] = aG[i];
+            toHash[3 * i + 3] = aHP[i];
+            rv.II[i] = scalarmultKey(Hi, xx[i]);
+            precomp(Ip[i].k, rv.II[i]);
+        }
+        size_t ndsRows = 3 * dsRows; //non Double Spendable Rows (see identity chains paper)
+        for (i = dsRows, ii = 0 ; i < rows ; i++, ii++) {
+            skpkGen(alpha[i], aG[i]); //need to save alphas for later..
+            toHash[ndsRows + 2 * ii + 1] = pk[index][i];
+            toHash[ndsRows + 2 * ii + 2] = aG[i];
+        }
+
+        c_old = hash_to_scalar(toHash);
+
+        
+        i = (index + 1) % cols;
+        if (i == 0) {
+            copy(rv.cc, c_old);
+        }
+        while (i != index) {
+
+            rv.ss[i] = skvGen(rows);            
+            sc_0(c.bytes);
+            for (j = 0; j < dsRows; j++) {
+                addKeys2(L, rv.ss[i][j], c_old, pk[i][j]);
+                hashToPoint(Hi, pk[i][j]);
+                addKeys3(R, rv.ss[i][j], Hi, c_old, Ip[j].k);
+                toHash[3 * j + 1] = pk[i][j];
+                toHash[3 * j + 2] = L; 
+                toHash[3 * j + 3] = R;
+            }
+            for (j = dsRows, ii = 0; j < rows; j++, ii++) {
+                addKeys2(L, rv.ss[i][j], c_old, pk[i][j]);
+                toHash[ndsRows + 2 * ii + 1] = pk[i][j];
+                toHash[ndsRows + 2 * ii + 2] = L;
+            }
+            c = hash_to_scalar(toHash);
+            copy(c_old, c);
+            i = (i + 1) % cols;
+            
+            if (i == 0) { 
+                copy(rv.cc, c_old);
+            }   
+        }
+        for (j = 0; j < rows; j++) {
+            sc_mulsub(rv.ss[index][j].bytes, c.bytes, xx[j].bytes, alpha[j].bytes);
+        }        
+        return rv;
+    }
+    
+    //Multilayered Spontaneous Anonymous Group Signatures (MLSAG signatures)
+    //This is a just slghtly more efficient version than the ones described below
+    //(will be explained in more detail in Ring Multisig paper
+    //These are aka MG signatutes in earlier drafts of the ring ct paper
+    // c.f. http://eprint.iacr.org/2015/1098 section 2. 
+    // keyImageV just does I[i] = xx[i] * Hash(xx[i] * G) for each i
+    // Gen creates a signature which proves that for some column in the keymatrix "pk"
+    //   the signer knows a secret key for each row in that column
+    // Ver verifies that the MG sig was created correctly            
+    bool MLSAG_Ver(key message, const keyM & pk, const mgSig & rv, size_t dsRows) {
+
+        size_t cols = pk.size();
+        CHECK_AND_ASSERT_MES(cols >= 2, false, "Error! What is c if cols = 1!");
+        size_t rows = pk[0].size();
+        CHECK_AND_ASSERT_MES(rows >= 1, false, "Empty pk");
+        for (size_t i = 1; i < cols; ++i) {
+          CHECK_AND_ASSERT_MES(pk[i].size() == rows, false, "pk is not rectangular");
+        }
+        CHECK_AND_ASSERT_MES(rv.II.size() == dsRows, false, "Bad II size");
+        CHECK_AND_ASSERT_MES(rv.ss.size() == cols, false, "Bad rv.ss size");
+        for (size_t i = 0; i < cols; ++i) {
+          CHECK_AND_ASSERT_MES(rv.ss[i].size() == rows, false, "rv.ss is not rectangular");
+        }
+        CHECK_AND_ASSERT_MES(dsRows <= rows, false, "Bad dsRows value");
+
+        size_t i = 0, j = 0, ii = 0;
+        key c,  L, R, Hi;
+        key c_old = copy(rv.cc);
+        vector<geDsmp> Ip(dsRows);
+        for (i = 0 ; i < dsRows ; i++) {
+            precomp(Ip[i].k, rv.II[i]);
+        }
+        size_t ndsRows = 3 * dsRows; //non Double Spendable Rows (see identity chains paper
+        keyV toHash(1 + 3 * dsRows + 2 * (rows - dsRows));
+        toHash[0] = message;
+        i = 0;
+        while (i < cols) {
+            sc_0(c.bytes);
+            for (j = 0; j < dsRows; j++) {
+                addKeys2(L, rv.ss[i][j], c_old, pk[i][j]);
+                hashToPoint(Hi, pk[i][j]);
+                addKeys3(R, rv.ss[i][j], Hi, c_old, Ip[j].k);
+                toHash[3 * j + 1] = pk[i][j];
+                toHash[3 * j + 2] = L; 
+                toHash[3 * j + 3] = R;
+            }
+            for (j = dsRows, ii = 0 ; j < rows ; j++, ii++) {
+                addKeys2(L, rv.ss[i][j], c_old, pk[i][j]);
+                toHash[ndsRows + 2 * ii + 1] = pk[i][j];
+                toHash[ndsRows + 2 * ii + 2] = L;
+            }
+            c = hash_to_scalar(toHash);
+            copy(c_old, c);
+            i = (i + 1);
+        }
+        sc_sub(c.bytes, c_old.bytes, rv.cc.bytes);
+        return sc_isnonzero(c.bytes) == 0;  
+    }
+    
+
+
+    //proveRange and verRange
+    //proveRange gives C, and mask such that \sumCi = C
+    //   c.f. http://eprint.iacr.org/2015/1098 section 5.1
+    //   and Ci is a commitment to either 0 or 2^i, i=0,...,63
+    //   thus this proves that "amount" is in [0, 2^64]
+    //   mask is a such that C = aG + bH, and b = amount
+    //verRange verifies that \sum Ci = C and that each Ci is a commitment to 0 or 2^i
+    rangeSig proveRange(key & C, key & mask, const xmr_amount & amount) {
+        sc_0(mask.bytes);
+        identity(C);
+        bits b;
+        d2b(b, amount);
+        rangeSig sig;
+        key64 ai;
+        key64 CiH;
+        int i = 0;
+        for (i = 0; i < ATOMS; i++) {
+            skGen(ai[i]);
+            if (b[i] == 0) {
+                scalarmultBase(sig.Ci[i], ai[i]);
+            }
+            if (b[i] == 1) {
+                addKeys1(sig.Ci[i], ai[i], H2[i]);
+            }
+            subKeys(CiH[i], sig.Ci[i], H2[i]);
+            sc_add(mask.bytes, mask.bytes, ai[i].bytes);
+            addKeys(C, C, sig.Ci[i]);
+        }
+        sig.asig = GenASNL(ai, sig.Ci, CiH, b);
+        return sig;
+    }
+
+    //proveRange and verRange
+    //proveRange gives C, and mask such that \sumCi = C
+    //   c.f. http://eprint.iacr.org/2015/1098 section 5.1
+    //   and Ci is a commitment to either 0 or 2^i, i=0,...,63
+    //   thus this proves that "amount" is in [0, 2^64]
+    //   mask is a such that C = aG + bH, and b = amount
+    //verRange verifies that \sum Ci = C and that each Ci is a commitment to 0 or 2^i
+    bool verRange(const key & C, const rangeSig & as) {
+        key64 CiH;
+        int i = 0;
+        key Ctmp = identity();
+        for (i = 0; i < 64; i++) {
+            subKeys(CiH[i], as.Ci[i], H2[i]);
+            addKeys(Ctmp, Ctmp, as.Ci[i]);
+        }
+        if (!equalKeys(C, Ctmp))
+          return false;
+        if (!VerASNL(as.Ci, CiH, as.asig))
+          return false;
+        return true;
+    }
+
+    key get_pre_mlsag_hash(const rctSig &rv)
+    {
+      keyV hashes;
+      hashes.push_back(rv.message);
+      crypto::hash h;
+      cryptonote::get_blob_hash(cryptonote::t_serializable_object_to_blob((const rctSigBase&)rv), h);
+      hashes.push_back(hash2rct(h));
+      keyV kv;
+      for (auto r: rv.p.rangeSigs)
+      {
+        for (size_t n = 0; n < 64; ++n)
+          kv.push_back(r.asig.L1[n]);
+        for (size_t n = 0; n < 64; ++n)
+          kv.push_back(r.asig.s2[n]);
+        kv.push_back(r.asig.s);
+        for (size_t n = 0; n < 64; ++n)
+          kv.push_back(r.Ci[n]);
+      }
+      hashes.push_back(cn_fast_hash(kv));
+
+      return cn_fast_hash(hashes);
+    }
+
+    //Ring-ct MG sigs
+    //Prove: 
+    //   c.f. http://eprint.iacr.org/2015/1098 section 4. definition 10. 
+    //   This does the MG sig on the "dest" part of the given key matrix, and 
+    //   the last row is the sum of input commitments from that column - sum output commitments
+    //   this shows that sum inputs = sum outputs
+    //Ver:    
+    //   verifies the above sig is created corretly
+    mgSig proveRctMG(const key &message, const ctkeyM & pubs, const ctkeyV & inSk, const ctkeyV &outSk, const ctkeyV & outPk, unsigned int index, key txnFeeKey) {
+        mgSig mg;
+        //setup vars
+        size_t cols = pubs.size();
+        CHECK_AND_ASSERT_THROW_MES(cols >= 1, "Empty pubs");
+        size_t rows = pubs[0].size();
+        CHECK_AND_ASSERT_THROW_MES(rows >= 1, "Empty pubs");
+        for (size_t i = 1; i < cols; ++i) {
+          CHECK_AND_ASSERT_THROW_MES(pubs[i].size() == rows, "pubs is not rectangular");
+        }
+        CHECK_AND_ASSERT_THROW_MES(inSk.size() == rows, "Bad inSk size");
+        CHECK_AND_ASSERT_THROW_MES(outSk.size() == outPk.size(), "Bad outSk/outPk size");
+
+        keyV sk(rows + 1);
+        keyV tmp(rows + 1);
+        size_t i = 0, j = 0;
+        for (i = 0; i < rows + 1; i++) {
+            sc_0(sk[i].bytes);
+            identity(tmp[i]);
+        }
+        keyM M(cols, tmp);
+        //create the matrix to mg sig
+        for (i = 0; i < cols; i++) {
+            M[i][rows] = identity();
+            for (j = 0; j < rows; j++) {
+                M[i][j] = pubs[i][j].dest;
+                addKeys(M[i][rows], M[i][rows], pubs[i][j].mask); //add input commitments in last row
+            }
+        }
+        sc_0(sk[rows].bytes);
+        for (j = 0; j < rows; j++) {
+            sk[j] = copy(inSk[j].dest);
+            sc_add(sk[rows].bytes, sk[rows].bytes, inSk[j].mask.bytes); //add masks in last row
+        }
+        for (i = 0; i < cols; i++) {
+            for (size_t j = 0; j < outPk.size(); j++) {
+                subKeys(M[i][rows], M[i][rows], outPk[j].mask); //subtract output Ci's in last row
+            }
+            //subtract txn fee output in last row
+            subKeys(M[i][rows], M[i][rows], txnFeeKey);
+        }
+        for (size_t j = 0; j < outPk.size(); j++) {
+            sc_sub(sk[rows].bytes, sk[rows].bytes, outSk[j].mask.bytes); //subtract output masks in last row..
+        }
+        return MLSAG_Gen(message, M, sk, index, rows);
+    }
+
+
+    //Ring-ct MG sigs Simple
+    //   Simple version for when we assume only
+    //       post rct inputs
+    //       here pubs is a vector of (P, C) length mixin
+    //   inSk is x, a_in corresponding to signing index
+    //       a_out, Cout is for the output commitment
+    //       index is the signing index..
+    mgSig proveRctMGSimple(const key &message, const ctkeyV & pubs, const ctkey & inSk, const key &a , const key &Cout, unsigned int index) {
+        mgSig mg;
+        //setup vars
+        size_t rows = 1;
+        size_t cols = pubs.size();
+        CHECK_AND_ASSERT_THROW_MES(cols >= 1, "Empty pubs");
+        keyV tmp(rows + 1);
+        keyV sk(rows + 1);
+        size_t i;
+        keyM M(cols, tmp);
+        for (i = 0; i < cols; i++) {
+            M[i][0] = pubs[i].dest;
+            subKeys(M[i][1], pubs[i].mask, Cout);
+            sk[0] = copy(inSk.dest);
+            sc_sub(sk[1].bytes, inSk.mask.bytes, a.bytes);  
+        }
+        return MLSAG_Gen(message, M, sk, index, rows);
+    }
+
+
+    //Ring-ct MG sigs
+    //Prove: 
+    //   c.f. http://eprint.iacr.org/2015/1098 section 4. definition 10. 
+    //   This does the MG sig on the "dest" part of the given key matrix, and 
+    //   the last row is the sum of input commitments from that column - sum output commitments
+    //   this shows that sum inputs = sum outputs
+    //Ver:    
+    //   verifies the above sig is created corretly
+    bool verRctMG(const mgSig &mg, const ctkeyM & pubs, const ctkeyV & outPk, key txnFeeKey, const key &message) {
+        //setup vars
+        size_t cols = pubs.size();
+        CHECK_AND_ASSERT_MES(cols >= 1, false, "Empty pubs");
+        size_t rows = pubs[0].size();
+        CHECK_AND_ASSERT_MES(rows >= 1, false, "Empty pubs");
+        for (size_t i = 1; i < cols; ++i) {
+          CHECK_AND_ASSERT_MES(pubs[i].size() == rows, false, "pubs is not rectangular");
+        }
+
+        keyV tmp(rows + 1);
+        size_t i = 0, j = 0;
+        for (i = 0; i < rows + 1; i++) {
+            identity(tmp[i]);
+        }
+        keyM M(cols, tmp);
+
+        //create the matrix to mg sig
+        for (j = 0; j < rows; j++) {
+            for (i = 0; i < cols; i++) {
+                M[i][j] = pubs[i][j].dest;
+                addKeys(M[i][rows], M[i][rows], pubs[i][j].mask); //add Ci in last row
+            }
+        }
+        for (i = 0; i < cols; i++) {
+            for (j = 0; j < outPk.size(); j++) {
+                subKeys(M[i][rows], M[i][rows], outPk[j].mask); //subtract output Ci's in last row
+            }
+            //subtract txn fee output in last row
+            subKeys(M[i][rows], M[i][rows], txnFeeKey);
+        }
+        return MLSAG_Ver(message, M, mg, rows);
+    }
+
+    //Ring-ct Simple MG sigs
+    //Ver: 
+    //This does a simplified version, assuming only post Rct
+    //inputs
+    bool verRctMGSimple(const key &message, const mgSig &mg, const ctkeyV & pubs, const key & C) {
+            //setup vars
+            size_t rows = 1;
+            size_t cols = pubs.size();
+            CHECK_AND_ASSERT_MES(cols >= 1, false, "Empty pubs");
+            keyV tmp(rows + 1);
+            size_t i;
+            keyM M(cols, tmp);
+            //create the matrix to mg sig
+            for (i = 0; i < cols; i++) {
+                    M[i][0] = pubs[i].dest;
+                    subKeys(M[i][1], pubs[i].mask, C);
+            }
+            //DP(C);
+            return MLSAG_Ver(message, M, mg, rows);
+    }
+
+    //These functions get keys from blockchain
+    //replace these when connecting blockchain
+    //getKeyFromBlockchain grabs a key from the blockchain at "reference_index" to mix with
+    //populateFromBlockchain creates a keymatrix with "mixin" columns and one of the columns is inPk
+    //   the return value are the key matrix, and the index where inPk was put (random).    
+    void getKeyFromBlockchain(ctkey & a, size_t reference_index) {
+        a.mask = pkGen();
+        a.dest = pkGen();
+    }
+
+    //These functions get keys from blockchain
+    //replace these when connecting blockchain
+    //getKeyFromBlockchain grabs a key from the blockchain at "reference_index" to mix with
+    //populateFromBlockchain creates a keymatrix with "mixin" + 1 columns and one of the columns is inPk
+    //   the return value are the key matrix, and the index where inPk was put (random).     
+    tuple<ctkeyM, xmr_amount> populateFromBlockchain(ctkeyV inPk, int mixin) {
+        int rows = inPk.size();
+        ctkeyM rv(mixin + 1, inPk);
+        int index = randXmrAmount(mixin);
+        int i = 0, j = 0;
+        for (i = 0; i <= mixin; i++) {
+            if (i != index) {
+                for (j = 0; j < rows; j++) {
+                    getKeyFromBlockchain(rv[i][j], (size_t)randXmrAmount);
+                }
+            }
+        }
+        return make_tuple(rv, index);
+    }
+
+    //These functions get keys from blockchain
+    //replace these when connecting blockchain
+    //getKeyFromBlockchain grabs a key from the blockchain at "reference_index" to mix with
+    //populateFromBlockchain creates a keymatrix with "mixin" columns and one of the columns is inPk
+    //   the return value are the key matrix, and the index where inPk was put (random).     
+    xmr_amount populateFromBlockchainSimple(ctkeyV & mixRing, const ctkey & inPk, int mixin) {
+        int index = randXmrAmount(mixin);
+        int i = 0;
+        for (i = 0; i <= mixin; i++) {
+            if (i != index) {
+                getKeyFromBlockchain(mixRing[i], (size_t)randXmrAmount(1000));
+            } else {
+                mixRing[i] = inPk;
+            }
+        }
+        return index;
+    }
+
+    //RingCT protocol
+    //genRct: 
+    //   creates an rctSig with all data necessary to verify the rangeProofs and that the signer owns one of the
+    //   columns that are claimed as inputs, and that the sum of inputs  = sum of outputs.
+    //   Also contains masked "amount" and "mask" so the receiver can see how much they received
+    //verRct:
+    //   verifies that all signatures (rangeProogs, MG sig, sum inputs = outputs) are correct
+    //decodeRct: (c.f. http://eprint.iacr.org/2015/1098 section 5.1.1)
+    //   uses the attached ecdh info to find the amounts represented by each output commitment 
+    //   must know the destination private key to find the correct amount, else will return a random number
+    //   Note: For txn fees, the last index in the amounts vector should contain that
+    //   Thus the amounts vector will be "one" longer than the destinations vectort
+    rctSig genRct(const key &message, const ctkeyV & inSk, const keyV & destinations, const vector<xmr_amount> & amounts, const ctkeyM &mixRing, const keyV &amount_keys, unsigned int index, ctkeyV &outSk) {
+        CHECK_AND_ASSERT_THROW_MES(amounts.size() == destinations.size() || amounts.size() == destinations.size() + 1, "Different number of amounts/destinations");
+        CHECK_AND_ASSERT_THROW_MES(index < mixRing.size(), "Bad index into mixRing");
+        for (size_t n = 0; n < mixRing.size(); ++n) {
+          CHECK_AND_ASSERT_THROW_MES(mixRing[n].size() == inSk.size(), "Bad mixRing size");
+        }
+
+        rctSig rv;
+        rv.type = RCTTypeFull;
+        rv.message = message;
+        rv.outPk.resize(destinations.size());
+        rv.p.rangeSigs.resize(destinations.size());
+        rv.ecdhInfo.resize(destinations.size());
+
+        size_t i = 0;
+        keyV masks(destinations.size()); //sk mask..
+        outSk.resize(destinations.size());
+        for (i = 0; i < destinations.size(); i++) {
+            //add destination to sig
+            rv.outPk[i].dest = copy(destinations[i]);
+            //compute range proof
+            rv.p.rangeSigs[i] = proveRange(rv.outPk[i].mask, outSk[i].mask, amounts[i]);
+            #ifdef DBG
+                CHECK_AND_ASSERT_THROW_MES(verRange(rv.outPk[i].mask, rv.p.rangeSigs[i]), "verRange failed on newly created proof");
+            #endif
+
+            //mask amount and mask
+            rv.ecdhInfo[i].mask = copy(outSk[i].mask);
+            rv.ecdhInfo[i].amount = d2h(amounts[i]);
+            ecdhEncode(rv.ecdhInfo[i], amount_keys[i]);
+
+        }
+
+        //set txn fee
+        if (amounts.size() > destinations.size())
+        {
+          rv.txnFee = amounts[destinations.size()];
+        }
+        else
+        {
+          rv.txnFee = 0;
+        }
+        key txnFeeKey = scalarmultH(d2h(rv.txnFee));
+
+        rv.mixRing = mixRing;
+        rv.p.MGs.push_back(proveRctMG(get_pre_mlsag_hash(rv), rv.mixRing, inSk, outSk, rv.outPk, index, txnFeeKey));
+        return rv;
+    }
+
+    rctSig genRct(const key &message, const ctkeyV & inSk, const ctkeyV  & inPk, const keyV & destinations, const vector<xmr_amount> & amounts, const keyV &amount_keys, const int mixin) {
+        unsigned int index;
+        ctkeyM mixRing;
+        ctkeyV outSk;
+        tie(mixRing, index) = populateFromBlockchain(inPk, mixin);
+        return genRct(message, inSk, destinations, amounts, mixRing, amount_keys, index, outSk);
+    }
+    
+    //RCT simple    
+    //for post-rct only
+    rctSig genRctSimple(const key &message, const ctkeyV & inSk, const keyV & destinations, const vector<xmr_amount> &inamounts, const vector<xmr_amount> &outamounts, xmr_amount txnFee, const ctkeyM & mixRing, const keyV &amount_keys, const std::vector<unsigned int> & index, ctkeyV &outSk) {
+        CHECK_AND_ASSERT_THROW_MES(inamounts.size() > 0, "Empty inamounts");
+        CHECK_AND_ASSERT_THROW_MES(inamounts.size() == inSk.size(), "Different number of inamounts/inSk");
+        CHECK_AND_ASSERT_THROW_MES(outamounts.size() == destinations.size(), "Different number of amounts/destinations");
+        CHECK_AND_ASSERT_THROW_MES(index.size() == inSk.size(), "Different number of index/inSk");
+        CHECK_AND_ASSERT_THROW_MES(mixRing.size() == inSk.size(), "Different number of mixRing/inSk");
+        for (size_t n = 0; n < mixRing.size(); ++n) {
+          CHECK_AND_ASSERT_THROW_MES(index[n] < mixRing[n].size(), "Bad index into mixRing");
+        }
+
+        rctSig rv;
+        rv.type = RCTTypeSimple;
+        rv.message = message;
+        rv.outPk.resize(destinations.size());
+        rv.p.rangeSigs.resize(destinations.size());
+        rv.ecdhInfo.resize(destinations.size());
+
+        size_t i;
+        keyV masks(destinations.size()); //sk mask..
+        outSk.resize(destinations.size());
+        key sumout = zero();
+        for (i = 0; i < destinations.size(); i++) {
+
+            //add destination to sig
+            rv.outPk[i].dest = copy(destinations[i]);
+            //compute range proof
+            rv.p.rangeSigs[i] = proveRange(rv.outPk[i].mask, outSk[i].mask, outamounts[i]);
+         #ifdef DBG
+             verRange(rv.outPk[i].mask, rv.p.rangeSigs[i]);
+         #endif
+         
+            sc_add(sumout.bytes, outSk[i].mask.bytes, sumout.bytes);
+
+            //mask amount and mask
+            rv.ecdhInfo[i].mask = copy(outSk[i].mask);
+            rv.ecdhInfo[i].amount = d2h(outamounts[i]);
+            ecdhEncode(rv.ecdhInfo[i], amount_keys[i]);
+        }
+            
+        //set txn fee
+        rv.txnFee = txnFee;
+//        TODO: unused ??
+//        key txnFeeKey = scalarmultH(d2h(rv.txnFee));
+        rv.mixRing = mixRing;
+        rv.pseudoOuts.resize(inamounts.size());
+        rv.p.MGs.resize(inamounts.size());
+        key sumpouts = zero(); //sum pseudoOut masks
+        keyV a(inamounts.size());
+        for (i = 0 ; i < inamounts.size() - 1; i++) {
+            skGen(a[i]);
+            sc_add(sumpouts.bytes, a[i].bytes, sumpouts.bytes);
+            genC(rv.pseudoOuts[i], a[i], inamounts[i]);
+        }
+        rv.mixRing = mixRing;
+        sc_sub(a[i].bytes, sumout.bytes, sumpouts.bytes);
+        genC(rv.pseudoOuts[i], a[i], inamounts[i]);
+        DP(rv.pseudoOuts[i]);
+
+        key full_message = get_pre_mlsag_hash(rv);
+        for (i = 0 ; i < inamounts.size(); i++) {
+            rv.p.MGs[i] = proveRctMGSimple(full_message, rv.mixRing[i], inSk[i], a[i], rv.pseudoOuts[i], index[i]);
+        }
+        return rv;
+    }
+
+    rctSig genRctSimple(const key &message, const ctkeyV & inSk, const ctkeyV & inPk, const keyV & destinations, const vector<xmr_amount> &inamounts, const vector<xmr_amount> &outamounts, const keyV &amount_keys, xmr_amount txnFee, unsigned int mixin) {
+        std::vector<unsigned int> index;
+        index.resize(inPk.size());
+        ctkeyM mixRing;
+        ctkeyV outSk;
+        mixRing.resize(inPk.size());
+        for (size_t i = 0; i < inPk.size(); ++i) {
+          mixRing[i].resize(mixin+1);
+          index[i] = populateFromBlockchainSimple(mixRing[i], inPk[i], mixin);
+        }
+        return genRctSimple(message, inSk, destinations, inamounts, outamounts, txnFee, mixRing, amount_keys, index, outSk);
+    }
+
+    //RingCT protocol
+    //genRct: 
+    //   creates an rctSig with all data necessary to verify the rangeProofs and that the signer owns one of the
+    //   columns that are claimed as inputs, and that the sum of inputs  = sum of outputs.
+    //   Also contains masked "amount" and "mask" so the receiver can see how much they received
+    //verRct:
+    //   verifies that all signatures (rangeProogs, MG sig, sum inputs = outputs) are correct
+    //decodeRct: (c.f. http://eprint.iacr.org/2015/1098 section 5.1.1)
+    //   uses the attached ecdh info to find the amounts represented by each output commitment 
+    //   must know the destination private key to find the correct amount, else will return a random number    
+    bool verRct(const rctSig & rv) {
+        CHECK_AND_ASSERT_MES(rv.type == RCTTypeFull, false, "verRct called on non-full rctSig");
+        CHECK_AND_ASSERT_MES(rv.outPk.size() == rv.p.rangeSigs.size(), false, "Mismatched sizes of outPk and rv.p.rangeSigs");
+        CHECK_AND_ASSERT_MES(rv.outPk.size() == rv.ecdhInfo.size(), false, "Mismatched sizes of outPk and rv.ecdhInfo");
+        CHECK_AND_ASSERT_MES(rv.p.MGs.size() == 1, false, "full rctSig has not one MG");
+
+        // some rct ops can throw
+        try
+        {
+          size_t i = 0;
+          bool tmp;
+          DP("range proofs verified?");
+          for (i = 0; i < rv.outPk.size(); i++) {
+              tmp = verRange(rv.outPk[i].mask, rv.p.rangeSigs[i]);
+              DP(tmp);
+              if (!tmp) {
+                LOG_ERROR("Range proof verification failed for input " << i);
+                return false;
+              }
+          }
+          //compute txn fee
+          key txnFeeKey = scalarmultH(d2h(rv.txnFee));
+          bool mgVerd = verRctMG(rv.p.MGs[0], rv.mixRing, rv.outPk, txnFeeKey, get_pre_mlsag_hash(rv));
+          DP("mg sig verified?");
+          DP(mgVerd);
+          if (!mgVerd) {
+            LOG_ERROR("MG signature verification failed");
+            return false;
+          }
+
+          return true;
+        }
+        catch(...)
+        {
+          return false;
+        }
+    }
+
+    //ver RingCT simple
+    //assumes only post-rct style inputs (at least for max anonymity)
+    bool verRctSimple(const rctSig & rv) {
+        size_t i = 0;
+
+        CHECK_AND_ASSERT_MES(rv.type == RCTTypeSimple, false, "verRctSimple called on non simple rctSig");
+        CHECK_AND_ASSERT_MES(rv.outPk.size() == rv.p.rangeSigs.size(), false, "Mismatched sizes of outPk and rv.p.rangeSigs");
+        CHECK_AND_ASSERT_MES(rv.outPk.size() == rv.ecdhInfo.size(), false, "Mismatched sizes of outPk and rv.ecdhInfo");
+        CHECK_AND_ASSERT_MES(rv.pseudoOuts.size() == rv.p.MGs.size(), false, "Mismatched sizes of rv.pseudoOuts and rv.p.MGs");
+        CHECK_AND_ASSERT_MES(rv.pseudoOuts.size() == rv.mixRing.size(), false, "Mismatched sizes of rv.pseudoOuts and mixRing");
+
+        key sumOutpks = identity();
+        for (i = 0; i < rv.outPk.size(); i++) {
+            if (!verRange(rv.outPk[i].mask, rv.p.rangeSigs[i])) {
+                LOG_ERROR("Range proof verified failed for input " << i);
+                return false;
+            }
+            addKeys(sumOutpks, sumOutpks, rv.outPk[i].mask);
+        }
+        DP(sumOutpks);
+        key txnFeeKey = scalarmultH(d2h(rv.txnFee));
+        addKeys(sumOutpks, txnFeeKey, sumOutpks);
+
+        bool tmpb = false;
+        key message = get_pre_mlsag_hash(rv);
+        key sumPseudoOuts = identity();
+        for (i = 0 ; i < rv.mixRing.size() ; i++) {
+            tmpb = verRctMGSimple(message, rv.p.MGs[i], rv.mixRing[i], rv.pseudoOuts[i]);
+            addKeys(sumPseudoOuts, sumPseudoOuts, rv.pseudoOuts[i]);
+            DP(tmpb);
+            if (!tmpb) {
+                LOG_ERROR("verRctMGSimple failed for input " << i);
+                return false;
+            }
+        }
+        DP(sumPseudoOuts);
+        
+        //check pseudoOuts vs Outs..
+        if (!equalKeys(sumPseudoOuts, sumOutpks)) {
+            LOG_ERROR("Sum check failed");
+            return false;
+        }
+
+        return true;
+    }
+
+    //RingCT protocol
+    //genRct: 
+    //   creates an rctSig with all data necessary to verify the rangeProofs and that the signer owns one of the
+    //   columns that are claimed as inputs, and that the sum of inputs  = sum of outputs.
+    //   Also contains masked "amount" and "mask" so the receiver can see how much they received
+    //verRct:
+    //   verifies that all signatures (rangeProogs, MG sig, sum inputs = outputs) are correct
+    //decodeRct: (c.f. http://eprint.iacr.org/2015/1098 section 5.1.1)
+    //   uses the attached ecdh info to find the amounts represented by each output commitment 
+    //   must know the destination private key to find the correct amount, else will return a random number    
+    xmr_amount decodeRct(const rctSig & rv, const key & sk, unsigned int i, key & mask) {
+        CHECK_AND_ASSERT_MES(rv.type == RCTTypeFull, false, "decodeRct called on non-full rctSig");
+        CHECK_AND_ASSERT_THROW_MES(rv.p.rangeSigs.size() > 0, "Empty rv.p.rangeSigs");
+        CHECK_AND_ASSERT_THROW_MES(rv.outPk.size() == rv.p.rangeSigs.size(), "Mismatched sizes of rv.outPk and rv.p.rangeSigs");
+        CHECK_AND_ASSERT_THROW_MES(i < rv.ecdhInfo.size(), "Bad index");
+
+        //mask amount and mask
+        ecdhTuple ecdh_info = rv.ecdhInfo[i];
+        ecdhDecode(ecdh_info, sk);
+        mask = ecdh_info.mask;
+        key amount = ecdh_info.amount;
+        key C = rv.outPk[i].mask;
+        DP("C");
+        DP(C);
+        key Ctmp;
+        addKeys2(Ctmp, mask, amount, H);
+        DP("Ctmp");
+        DP(Ctmp);
+        if (equalKeys(C, Ctmp) == false) {
+            CHECK_AND_ASSERT_THROW_MES(false, "warning, amount decoded incorrectly, will be unable to spend");
+        }
+        return h2d(amount);
+    }
+
+    xmr_amount decodeRct(const rctSig & rv, const key & sk, unsigned int i) {
+      key mask;
+      return decodeRct(rv, sk, i, mask);
+    }
+
+    xmr_amount decodeRctSimple(const rctSig & rv, const key & sk, unsigned int i, key &mask) {
+        CHECK_AND_ASSERT_MES(rv.type == RCTTypeSimple, false, "decodeRct called on non simple rctSig");
+        CHECK_AND_ASSERT_THROW_MES(rv.p.rangeSigs.size() > 0, "Empty rv.p.rangeSigs");
+        CHECK_AND_ASSERT_THROW_MES(rv.outPk.size() == rv.p.rangeSigs.size(), "Mismatched sizes of rv.outPk and rv.p.rangeSigs");
+        CHECK_AND_ASSERT_THROW_MES(i < rv.ecdhInfo.size(), "Bad index");
+
+        //mask amount and mask
+        ecdhTuple ecdh_info = rv.ecdhInfo[i];
+        ecdhDecode(ecdh_info, sk);
+        mask = ecdh_info.mask;
+        key amount = ecdh_info.amount;
+        key C = rv.outPk[i].mask;
+        DP("C");
+        DP(C);
+        key Ctmp;
+        addKeys2(Ctmp, mask, amount, H);
+        DP("Ctmp");
+        DP(Ctmp);
+        if (equalKeys(C, Ctmp) == false) {
+            CHECK_AND_ASSERT_THROW_MES(false, "warning, amount decoded incorrectly, will be unable to spend");
+        }
+        return h2d(amount);
+    }
+
+    xmr_amount decodeRctSimple(const rctSig & rv, const key & sk, unsigned int i) {
+      key mask;
+      return decodeRctSimple(rv, sk, i, mask);
+    }
+}

src/ringct/rctSigs.h

@@ -0,0 +1,150 @@
+// Copyright (c) 2016, Monero Research Labs
+//
+// Author: Shen Noether <shen.noether@gmx.com>
+//
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+//
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+//
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+//
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+#pragma once
+
+//#define DBG
+
+#ifndef RCTSIGS_H
+#define RCTSIGS_H
+
+#include <cstddef>
+#include <mutex>
+#include <vector>
+#include <tuple>
+
+#include "crypto/generic-ops.h"
+
+extern "C" {
+#include "crypto/random.h"
+#include "crypto/keccak.h"
+}
+#include "crypto/crypto.h"
+
+
+#include "rctTypes.h"
+#include "rctOps.h"
+
+//Define this flag when debugging to get additional info on the console
+#ifdef DBG
+#define DP(x) dp(x)
+#else
+#define DP(x)
+#endif
+
+
+
+using namespace std;
+using namespace crypto;
+
+namespace rct {
+
+    //Schnorr Non-linkable
+    //Gen Gives a signature (L1, s1, s2) proving that the sender knows "x" such that xG = one of P1 or P2
+    //Ver Verifies that signer knows an "x" such that xG = one of P1 or P2
+    //These are called in the below ASNL sig generation
+    void GenSchnorrNonLinkable(key & L1, key & s1, key & s2, const key & x, const key & P1, const key & P2, int index);
+    bool VerSchnorrNonLinkable(const key & P1, const key & P2, const key & L1, const key & s1, const key & s2);
+
+    //Aggregate Schnorr Non-linkable Ring Signature (ASNL)
+    // c.f. http://eprint.iacr.org/2015/1098 section 5.
+    // These are used in range proofs (alternatively Borromean could be used)
+    // Gen gives a signature which proves the signer knows, for each i,
+    //   an x[i] such that x[i]G = one of P1[i] or P2[i]
+    // Ver Verifies the signer knows a key for one of P1[i], P2[i] at each i
+    asnlSig GenASNL(key64 x, key64 P1, key64 P2, bits indices);
+    bool VerASNL(const key64 P1, const key64 P2, const asnlSig &as);
+
+    //Multilayered Spontaneous Anonymous Group Signatures (MLSAG signatures)
+    //These are aka MG signatutes in earlier drafts of the ring ct paper
+    // c.f. http://eprint.iacr.org/2015/1098 section 2.
+    // keyImageV just does I[i] = xx[i] * HashToPoint(xx[i] * G) for each i
+    // Gen creates a signature which proves that for some column in the keymatrix "pk"
+    //   the signer knows a secret key for each row in that column
+    // Ver verifies that the MG sig was created correctly
+    keyV keyImageV(const keyV &xx);
+    mgSig MLSAG_Gen(key message, const keyM & pk, const keyV & xx, const unsigned int index, size_t dsRows);
+    bool MLSAG_Ver(key message, const keyM &pk, const mgSig &sig, size_t dsRows);
+    //mgSig MLSAG_Gen_Old(const keyM & pk, const keyV & xx, const int index);
+
+    //proveRange and verRange
+    //proveRange gives C, and mask such that \sumCi = C
+    //   c.f. http://eprint.iacr.org/2015/1098 section 5.1
+    //   and Ci is a commitment to either 0 or 2^i, i=0,...,63
+    //   thus this proves that "amount" is in [0, 2^64]
+    //   mask is a such that C = aG + bH, and b = amount
+    //verRange verifies that \sum Ci = C and that each Ci is a commitment to 0 or 2^i
+    rangeSig proveRange(key & C, key & mask, const xmr_amount & amount);
+    bool verRange(const key & C, const rangeSig & as);
+
+    //Ring-ct MG sigs
+    //Prove:
+    //   c.f. http://eprint.iacr.org/2015/1098 section 4. definition 10.
+    //   This does the MG sig on the "dest" part of the given key matrix, and
+    //   the last row is the sum of input commitments from that column - sum output commitments
+    //   this shows that sum inputs = sum outputs
+    //Ver:
+    //   verifies the above sig is created corretly
+    mgSig proveRctMG(const ctkeyM & pubs, const ctkeyV & inSk, const keyV &outMasks, const ctkeyV & outPk, unsigned int index, key txnFee, const key &message);
+    mgSig proveRctMGSimple(const key & message, const ctkeyV & pubs, const ctkey & inSk, const key &a , const key &Cout, unsigned int index);
+    bool verRctMG(const mgSig &mg, const ctkeyM & pubs, const ctkeyV & outPk, key txnFee, const key &message);
+    bool verRctMGSimple(const key &message, const mgSig &mg, const ctkeyV & pubs, const key & C);
+
+    //These functions get keys from blockchain
+    //replace these when connecting blockchain
+    //getKeyFromBlockchain grabs a key from the blockchain at "reference_index" to mix with
+    //populateFromBlockchain creates a keymatrix with "mixin" columns and one of the columns is inPk
+    //   the return value are the key matrix, and the index where inPk was put (random).
+    void getKeyFromBlockchain(ctkey & a, size_t reference_index);
+    tuple<ctkeyM, xmr_amount> populateFromBlockchain(ctkeyV inPk, int mixin);
+
+    //RingCT protocol
+    //genRct:
+    //   creates an rctSig with all data necessary to verify the rangeProofs and that the signer owns one of the
+    //   columns that are claimed as inputs, and that the sum of inputs  = sum of outputs.
+    //   Also contains masked "amount" and "mask" so the receiver can see how much they received
+    //verRct:
+    //   verifies that all signatures (rangeProogs, MG sig, sum inputs = outputs) are correct
+    //decodeRct: (c.f. http://eprint.iacr.org/2015/1098 section 5.1.1)
+    //   uses the attached ecdh info to find the amounts represented by each output commitment
+    //   must know the destination private key to find the correct amount, else will return a random number
+    rctSig genRct(const key &message, const ctkeyV & inSk, const keyV & destinations, const vector<xmr_amount> & amounts, const ctkeyM &mixRing, const keyV &amount_keys, unsigned int index, ctkeyV &outSk);
+    rctSig genRct(const key &message, const ctkeyV & inSk, const ctkeyV  & inPk, const keyV & destinations, const vector<xmr_amount> & amounts, const keyV &amount_keys, const int mixin);
+    rctSig genRctSimple(const key & message, const ctkeyV & inSk, const ctkeyV & inPk, const keyV & destinations, const vector<xmr_amount> & inamounts, const vector<xmr_amount> & outamounts, const keyV &amount_keys, xmr_amount txnFee, unsigned int mixin);
+    rctSig genRctSimple(const key & message, const ctkeyV & inSk, const keyV & destinations, const vector<xmr_amount> & inamounts, const vector<xmr_amount> & outamounts, xmr_amount txnFee, const ctkeyM & mixRing, const keyV &amount_keys, const std::vector<unsigned int> & index, ctkeyV &outSk);
+    bool verRct(const rctSig & rv);
+    bool verRctSimple(const rctSig & rv);
+    xmr_amount decodeRct(const rctSig & rv, const key & sk, unsigned int i, key & mask);
+    xmr_amount decodeRct(const rctSig & rv, const key & sk, unsigned int i);
+    xmr_amount decodeRctSimple(const rctSig & rv, const key & sk, unsigned int i, key & mask);
+    xmr_amount decodeRctSimple(const rctSig & rv, const key & sk, unsigned int i);
+}
+#endif  /* RCTSIGS_H */
+

src/ringct/rctTypes.cpp

@@ -0,0 +1,208 @@
+// Copyright (c) 2016, Monero Research Labs
+//
+// Author: Shen Noether <shen.noether@gmx.com>
+// 
+// All rights reserved.
+// 
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+// 
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+// 
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+// 
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+// 
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+#include "rctTypes.h"
+using namespace crypto;
+using namespace std;
+
+namespace rct {
+
+    //dp 
+    //Debug printing for the above types
+    //Actually use DP(value) and #define DBG    
+    
+    void dp(key a) {
+        int j = 0;
+        printf("\"");
+        for (j = 0; j < 32; j++) {
+            printf("%02x", (unsigned char)a.bytes[j]);
+        }
+        printf("\"");
+        printf("\n");
+    }
+
+    void dp(bool a) {
+        printf(" ... %s ... ", a ? "true" : "false");
+        printf("\n");
+    }
+
+    void dp(const char * a, int l) {
+        int j = 0;
+        printf("\"");
+        for (j = 0; j < l; j++) {
+            printf("%02x", (unsigned char)a[j]);
+        }
+        printf("\"");
+        printf("\n");
+    }
+    void dp(keyV a) {
+        size_t j = 0;
+        printf("[");
+        for (j = 0; j < a.size(); j++) {
+            dp(a[j]);
+            if (j < a.size() - 1) {
+                printf(",");
+            }
+        }
+        printf("]");
+        printf("\n");
+    }
+    void dp(keyM a) {
+        size_t j = 0;
+        printf("[");
+        for (j = 0; j < a.size(); j++) {
+            dp(a[j]);
+            if (j < a.size() - 1) {
+                printf(",");
+            }
+        }
+        printf("]");
+        printf("\n");
+    }
+    void dp(xmr_amount vali) {
+        printf("x: ");
+        std::cout << vali;
+        printf("\n\n");
+    }
+
+    void dp(int vali) {
+        printf("x: %d\n", vali);
+        printf("\n");
+    }
+    void dp(bits amountb) {
+        for (int i = 0; i < 64; i++) {
+            printf("%d", amountb[i]);
+        }
+        printf("\n");
+
+    }
+
+    void dp(const char * st) {
+        printf("%s\n", st);
+    }
+
+    //Various Conversions 
+    
+    //uint long long to 32 byte key
+    void d2h(key & amounth, const xmr_amount in) {
+        sc_0(amounth.bytes);
+        xmr_amount val = in;
+        int i = 0;
+        while (val != 0) {
+            amounth[i] = (unsigned char)(val & 0xFF);
+            i++;
+            val /= (xmr_amount)256;
+        }
+    }
+    
+    //uint long long to 32 byte key
+    key d2h(const xmr_amount in) {
+        key amounth;
+        sc_0(amounth.bytes);
+        xmr_amount val = in;
+        int i = 0;
+        while (val != 0) {
+            amounth[i] = (unsigned char)(val & 0xFF);
+            i++;
+            val /= (xmr_amount)256;
+        }
+        return amounth;
+    }
+
+    //uint long long to int[64]
+    void d2b(bits  amountb, xmr_amount val) {
+        int i = 0;
+        while (val != 0) {
+            amountb[i] = val & 1;
+            i++;
+            val >>= 1;
+        }
+        while (i < 64) {
+            amountb[i] = 0;
+            i++;
+        }
+    }
+    
+    //32 byte key to uint long long
+    // if the key holds a value > 2^64
+    // then the value in the first 8 bytes is returned    
+    xmr_amount h2d(const key & test) {
+        xmr_amount vali = 0;
+        int j = 0;
+        for (j = 7; j >= 0; j--) {
+            vali = (xmr_amount)(vali * 256 + (unsigned char)test.bytes[j]);
+        }
+        return vali;
+    }
+    
+    //32 byte key to int[64]
+    void h2b(bits amountb2, const key & test) {
+        int val = 0, i = 0, j = 0;
+        for (j = 0; j < 8; j++) {
+            val = (unsigned char)test.bytes[j];
+            i = 8 * j;
+            while (val != 0) {
+                amountb2[i] = val & 1;
+                i++;
+                val >>= 1;
+            }
+            while (i < 8 * (j + 1)) {
+                amountb2[i] = 0;
+            }
+        }
+    }
+    
+    //int[64] to 32 byte key
+    void b2h(key & amountdh, const bits amountb2) {
+        int byte, i, j;
+        for (j = 0; j < 8; j++) {
+            byte = 0;
+            i = 8 * j;
+            for (i = 7; i > -1; i--) {
+                byte = byte * 2 + amountb2[8 * j + i];
+            }
+            amountdh[j] = (unsigned char)byte;
+        }
+        for (j = 8; j < 32; j++) {
+            amountdh[j] = (unsigned char)(0x00);
+        }
+    }
+    
+    //int[64] to uint long long
+    xmr_amount b2d(bits amountb) {
+        xmr_amount vali = 0;
+        int j = 0;
+        for (j = 63; j >= 0; j--) {
+            vali = (xmr_amount)(vali * 2 + amountb[j]);
+        }
+        return vali;
+    }
+
+}

src/ringct/rctTypes.h

@@ -0,0 +1,405 @@
+// Copyright (c) 2016, Monero Research Labs
+//
+// Author: Shen Noether <shen.noether@gmx.com>
+//
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+//
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+//
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+//
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+#pragma once
+#ifndef RCT_TYPES_H
+#define RCT_TYPES_H
+
+#include <cstddef>
+#include <mutex>
+#include <vector>
+#include <tuple>
+#include <iostream>
+#include <cinttypes>
+
+extern "C" {
+#include "crypto/generic-ops.h"
+#include "crypto/crypto-ops.h"
+#include "crypto/random.h"
+#include "crypto/keccak.h"
+}
+#include "crypto/crypto.h"
+
+#include "serialization/serialization.h"
+#include "serialization/debug_archive.h"
+#include "serialization/binary_archive.h"
+#include "serialization/json_archive.h"
+
+
+//Define this flag when debugging to get additional info on the console
+#ifdef DBG
+#define DP(x) dp(x)
+#else
+#define DP(x)
+#endif
+
+//atomic units of moneros
+#define ATOMS 64
+
+//for printing large ints
+
+using namespace std;
+using namespace crypto;
+
+//Namespace specifically for ring ct code
+namespace rct {
+    //basic ops containers
+    typedef unsigned char * Bytes;
+
+    // Can contain a secret or public key
+    //  similar to secret_key / public_key of crypto-ops,
+    //  but uses unsigned chars,
+    //  also includes an operator for accessing the i'th byte.
+    struct key {
+        unsigned char & operator[](int i) {
+            return bytes[i];
+        }
+        unsigned char operator[](int i) const {
+            return bytes[i];
+        }
+        bool operator==(const key &k) const { return !memcmp(bytes, k.bytes, sizeof(bytes)); }
+        unsigned char bytes[32];
+    };
+    typedef vector<key> keyV; //vector of keys
+    typedef vector<keyV> keyM; //matrix of keys (indexed by column first)
+
+    //containers For CT operations
+    //if it's  representing a private ctkey then "dest" contains the secret key of the address
+    // while "mask" contains a where C = aG + bH is CT pedersen commitment and b is the amount
+    // (store b, the amount, separately
+    //if it's representing a public ctkey, then "dest" = P the address, mask = C the commitment
+    struct ctkey {
+        key dest;
+        key mask; //C here if public
+    };
+    typedef vector<ctkey> ctkeyV;
+    typedef vector<ctkeyV> ctkeyM;
+
+    //data for passing the amount to the receiver secretly
+    // If the pedersen commitment to an amount is C = aG + bH,
+    // "mask" contains a 32 byte key a
+    // "amount" contains a hex representation (in 32 bytes) of a 64 bit number
+    // "senderPk" is not the senders actual public key, but a one-time public key generated for
+    // the purpose of the ECDH exchange
+    struct ecdhTuple {
+        key mask;
+        key amount;
+        key senderPk;
+
+        BEGIN_SERIALIZE_OBJECT()
+          FIELD(mask)
+          FIELD(amount)
+          // FIELD(senderPk) // not serialized, as we do not use it in monero currently
+        END_SERIALIZE()
+    };
+
+    //containers for representing amounts
+    typedef uint64_t xmr_amount;
+    typedef unsigned int bits[ATOMS];
+    typedef key key64[64];
+
+    //just contains the necessary keys to represent asnlSigs
+    //c.f. http://eprint.iacr.org/2015/1098
+    struct asnlSig {
+        key64 L1;
+        key64 s2;
+        key s;
+    };
+  
+    //Container for precomp
+    struct geDsmp {
+        ge_dsmp k;
+    };
+    
+    //just contains the necessary keys to represent MLSAG sigs
+    //c.f. http://eprint.iacr.org/2015/1098
+    struct mgSig {
+        keyM ss;
+        key cc;
+        keyV II;
+
+        BEGIN_SERIALIZE_OBJECT()
+            FIELD(ss)
+            FIELD(cc)
+            // FIELD(II) - not serialized, it can be reconstructed
+        END_SERIALIZE()
+    };
+    //contains the data for an asnl sig
+    // also contains the "Ci" values such that
+    // \sum Ci = C
+    // and the signature proves that each Ci is either
+    // a Pedersen commitment to 0 or to 2^i
+    //thus proving that C is in the range of [0, 2^64]
+    struct rangeSig {
+        asnlSig asig;
+        key64 Ci;
+
+        BEGIN_SERIALIZE_OBJECT()
+            FIELD(asig)
+            FIELD(Ci)
+        END_SERIALIZE()
+    };
+    //A container to hold all signatures necessary for RingCT
+    // rangeSigs holds all the rangeproof data of a transaction
+    // MG holds the MLSAG signature of a transaction
+    // mixRing holds all the public keypairs (P, C) for a transaction
+    // ecdhInfo holds an encoded mask / amount to be passed to each receiver
+    // outPk contains public keypairs which are destinations (P, C),
+    //  P = address, C = commitment to amount
+    enum {
+      RCTTypeNull = 0,
+      RCTTypeFull = 1,
+      RCTTypeSimple = 2,
+    };
+    struct rctSigBase {
+        uint8_t type;
+        key message;
+        ctkeyM mixRing; //the set of all pubkeys / copy
+        //pairs that you mix with
+        keyV pseudoOuts; //C - for simple rct
+        vector<ecdhTuple> ecdhInfo;
+        ctkeyV outPk;
+        xmr_amount txnFee; // contains b
+
+        BEGIN_SERIALIZE()
+            FIELD(type)
+            if (type == RCTTypeNull)
+              return true;
+            // FIELD(message) - not serialized, it can be reconstructed
+            // FIELD(mixRing) - not serialized, it can be reconstructed
+            if (type == RCTTypeSimple)
+              FIELD(pseudoOuts)
+            FIELD(ecdhInfo)
+            if (typename Archive<W>::is_saving()) {
+              keyV outPk(this->outPk.size());
+              for (size_t n = 0; n < outPk.size(); ++n)
+                outPk[n] = this->outPk[n].mask;
+              FIELD(outPk)
+            }
+            else {
+              keyV outPk;
+              FIELD(outPk)
+              this->outPk.resize(outPk.size());
+              for (size_t n = 0; n < outPk.size(); ++n)
+                this->outPk[n].mask = outPk[n];
+            }
+            VARINT_FIELD(txnFee)
+        END_SERIALIZE()
+    };
+    struct rctSigPrunable {
+        vector<rangeSig> rangeSigs;
+        vector<mgSig> MGs; // simple rct has N, full has 1
+
+        BEGIN_SERIALIZE()
+            FIELD(rangeSigs)
+            FIELD(MGs)
+        END_SERIALIZE()
+    };
+    struct rctSig: public rctSigBase {
+        rctSigPrunable p;
+
+        BEGIN_SERIALIZE_OBJECT()
+            FIELDS(*static_cast<rctSigBase *>(this))
+            if (type == RCTTypeNull)
+              return true;
+            FIELDS(p);
+        END_SERIALIZE()
+    };
+
+    //other basepoint H = toPoint(cn_fast_hash(G)), G the basepoint
+    static const key H = { {0x8b, 0x65, 0x59, 0x70, 0x15, 0x37, 0x99, 0xaf, 0x2a, 0xea, 0xdc, 0x9f, 0xf1, 0xad, 0xd0, 0xea, 0x6c, 0x72, 0x51, 0xd5, 0x41, 0x54, 0xcf, 0xa9, 0x2c, 0x17, 0x3a, 0x0d, 0xd3, 0x9c, 0x1f, 0x94} };
+
+    //H2 contains 2^i H in each index, i.e. H, 2H, 4H, 8H, ...
+    //This is used for the range proofG
+    //You can regenerate this by running python2 Test.py HPow2 in the MiniNero repo
+    static const key64 H2 = {{0x8b, 0x65, 0x59, 0x70, 0x15, 0x37, 0x99, 0xaf, 0x2a, 0xea, 0xdc, 0x9f, 0xf1, 0xad, 0xd0, 0xea, 0x6c, 0x72, 0x51, 0xd5, 0x41, 0x54, 0xcf, 0xa9, 0x2c, 0x17, 0x3a, 0x0d, 0xd3, 0x9c, 0x1f, 0x94},
+    {0x8f, 0xaa, 0x44, 0x8a, 0xe4, 0xb3, 0xe2, 0xbb, 0x3d, 0x4d, 0x13, 0x09, 0x09, 0xf5, 0x5f, 0xcd, 0x79, 0x71, 0x1c, 0x1c, 0x83, 0xcd, 0xbc, 0xca, 0xdd, 0x42, 0xcb, 0xe1, 0x51, 0x5e, 0x87, 0x12},
+    {0x12, 0xa7, 0xd6, 0x2c, 0x77, 0x91, 0x65, 0x4a, 0x57, 0xf3, 0xe6, 0x76, 0x94, 0xed, 0x50, 0xb4, 0x9a, 0x7d, 0x9e, 0x3f, 0xc1, 0xe4, 0xc7, 0xa0, 0xbd, 0xe2, 0x9d, 0x18, 0x7e, 0x9c, 0xc7, 0x1d},
+    {0x78, 0x9a, 0xb9, 0x93, 0x4b, 0x49, 0xc4, 0xf9, 0xe6, 0x78, 0x5c, 0x6d, 0x57, 0xa4, 0x98, 0xb3, 0xea, 0xd4, 0x43, 0xf0, 0x4f, 0x13, 0xdf, 0x11, 0x0c, 0x54, 0x27, 0xb4, 0xf2, 0x14, 0xc7, 0x39},
+    {0x77, 0x1e, 0x92, 0x99, 0xd9, 0x4f, 0x02, 0xac, 0x72, 0xe3, 0x8e, 0x44, 0xde, 0x56, 0x8a, 0xc1, 0xdc, 0xb2, 0xed, 0xc6, 0xed, 0xb6, 0x1f, 0x83, 0xca, 0x41, 0x8e, 0x10, 0x77, 0xce, 0x3d, 0xe8},
+    {0x73, 0xb9, 0x6d, 0xb4, 0x30, 0x39, 0x81, 0x9b, 0xda, 0xf5, 0x68, 0x0e, 0x5c, 0x32, 0xd7, 0x41, 0x48, 0x88, 0x84, 0xd1, 0x8d, 0x93, 0x86, 0x6d, 0x40, 0x74, 0xa8, 0x49, 0x18, 0x2a, 0x8a, 0x64},
+    {0x8d, 0x45, 0x8e, 0x1c, 0x2f, 0x68, 0xeb, 0xeb, 0xcc, 0xd2, 0xfd, 0x5d, 0x37, 0x9f, 0x5e, 0x58, 0xf8, 0x13, 0x4d, 0xf3, 0xe0, 0xe8, 0x8c, 0xad, 0x3d, 0x46, 0x70, 0x10, 0x63, 0xa8, 0xd4, 0x12},
+    {0x09, 0x55, 0x1e, 0xdb, 0xe4, 0x94, 0x41, 0x8e, 0x81, 0x28, 0x44, 0x55, 0xd6, 0x4b, 0x35, 0xee, 0x8a, 0xc0, 0x93, 0x06, 0x8a, 0x5f, 0x16, 0x1f, 0xa6, 0x63, 0x75, 0x59, 0x17, 0x7e, 0xf4, 0x04},
+    {0xd0, 0x5a, 0x88, 0x66, 0xf4, 0xdf, 0x8c, 0xee, 0x1e, 0x26, 0x8b, 0x1d, 0x23, 0xa4, 0xc5, 0x8c, 0x92, 0xe7, 0x60, 0x30, 0x97, 0x86, 0xcd, 0xac, 0x0f, 0xed, 0xa1, 0xd2, 0x47, 0xa9, 0xc9, 0xa7},
+    {0x55, 0xcd, 0xaa, 0xd5, 0x18, 0xbd, 0x87, 0x1d, 0xd1, 0xeb, 0x7b, 0xc7, 0x02, 0x3e, 0x1d, 0xc0, 0xfd, 0xf3, 0x33, 0x98, 0x64, 0xf8, 0x8f, 0xdd, 0x2d, 0xe2, 0x69, 0xfe, 0x9e, 0xe1, 0x83, 0x2d},
+    {0xe7, 0x69, 0x7e, 0x95, 0x1a, 0x98, 0xcf, 0xd5, 0x71, 0x2b, 0x84, 0xbb, 0xe5, 0xf3, 0x4e, 0xd7, 0x33, 0xe9, 0x47, 0x3f, 0xcb, 0x68, 0xed, 0xa6, 0x6e, 0x37, 0x88, 0xdf, 0x19, 0x58, 0xc3, 0x06},
+    {0xf9, 0x2a, 0x97, 0x0b, 0xae, 0x72, 0x78, 0x29, 0x89, 0xbf, 0xc8, 0x3a, 0xdf, 0xaa, 0x92, 0xa4, 0xf4, 0x9c, 0x7e, 0x95, 0x91, 0x8b, 0x3b, 0xba, 0x3c, 0xdc, 0x7f, 0xe8, 0x8a, 0xcc, 0x8d, 0x47},
+    {0x1f, 0x66, 0xc2, 0xd4, 0x91, 0xd7, 0x5a, 0xf9, 0x15, 0xc8, 0xdb, 0x6a, 0x6d, 0x1c, 0xb0, 0xcd, 0x4f, 0x7d, 0xdc, 0xd5, 0xe6, 0x3d, 0x3b, 0xa9, 0xb8, 0x3c, 0x86, 0x6c, 0x39, 0xef, 0x3a, 0x2b},
+    {0x3e, 0xec, 0x98, 0x84, 0xb4, 0x3f, 0x58, 0xe9, 0x3e, 0xf8, 0xde, 0xea, 0x26, 0x00, 0x04, 0xef, 0xea, 0x2a, 0x46, 0x34, 0x4f, 0xc5, 0x96, 0x5b, 0x1a, 0x7d, 0xd5, 0xd1, 0x89, 0x97, 0xef, 0xa7},
+    {0xb2, 0x9f, 0x8f, 0x0c, 0xcb, 0x96, 0x97, 0x7f, 0xe7, 0x77, 0xd4, 0x89, 0xd6, 0xbe, 0x9e, 0x7e, 0xbc, 0x19, 0xc4, 0x09, 0xb5, 0x10, 0x35, 0x68, 0xf2, 0x77, 0x61, 0x1d, 0x7e, 0xa8, 0x48, 0x94},
+    {0x56, 0xb1, 0xf5, 0x12, 0x65, 0xb9, 0x55, 0x98, 0x76, 0xd5, 0x8d, 0x24, 0x9d, 0x0c, 0x14, 0x6d, 0x69, 0xa1, 0x03, 0x63, 0x66, 0x99, 0x87, 0x4d, 0x3f, 0x90, 0x47, 0x35, 0x50, 0xfe, 0x3f, 0x2c},
+    {0x1d, 0x7a, 0x36, 0x57, 0x5e, 0x22, 0xf5, 0xd1, 0x39, 0xff, 0x9c, 0xc5, 0x10, 0xfa, 0x13, 0x85, 0x05, 0x57, 0x6b, 0x63, 0x81, 0x5a, 0x94, 0xe4, 0xb0, 0x12, 0xbf, 0xd4, 0x57, 0xca, 0xaa, 0xda},
+    {0xd0, 0xac, 0x50, 0x7a, 0x86, 0x4e, 0xcd, 0x05, 0x93, 0xfa, 0x67, 0xbe, 0x7d, 0x23, 0x13, 0x43, 0x92, 0xd0, 0x0e, 0x40, 0x07, 0xe2, 0x53, 0x48, 0x78, 0xd9, 0xb2, 0x42, 0xe1, 0x0d, 0x76, 0x20},
+    {0xf6, 0xc6, 0x84, 0x0b, 0x9c, 0xf1, 0x45, 0xbb, 0x2d, 0xcc, 0xf8, 0x6e, 0x94, 0x0b, 0xe0, 0xfc, 0x09, 0x8e, 0x32, 0xe3, 0x10, 0x99, 0xd5, 0x6f, 0x7f, 0xe0, 0x87, 0xbd, 0x5d, 0xeb, 0x50, 0x94},
+    {0x28, 0x83, 0x1a, 0x33, 0x40, 0x07, 0x0e, 0xb1, 0xdb, 0x87, 0xc1, 0x2e, 0x05, 0x98, 0x0d, 0x5f, 0x33, 0xe9, 0xef, 0x90, 0xf8, 0x3a, 0x48, 0x17, 0xc9, 0xf4, 0xa0, 0xa3, 0x32, 0x27, 0xe1, 0x97},
+    {0x87, 0x63, 0x22, 0x73, 0xd6, 0x29, 0xcc, 0xb7, 0xe1, 0xed, 0x1a, 0x76, 0x8f, 0xa2, 0xeb, 0xd5, 0x17, 0x60, 0xf3, 0x2e, 0x1c, 0x0b, 0x86, 0x7a, 0x5d, 0x36, 0x8d, 0x52, 0x71, 0x05, 0x5c, 0x6e},
+    {0x5c, 0x7b, 0x29, 0x42, 0x43, 0x47, 0x96, 0x4d, 0x04, 0x27, 0x55, 0x17, 0xc5, 0xae, 0x14, 0xb6, 0xb5, 0xea, 0x27, 0x98, 0xb5, 0x73, 0xfc, 0x94, 0xe6, 0xe4, 0x4a, 0x53, 0x21, 0x60, 0x0c, 0xfb},
+    {0xe6, 0x94, 0x50, 0x42, 0xd7, 0x8b, 0xc2, 0xc3, 0xbd, 0x6e, 0xc5, 0x8c, 0x51, 0x1a, 0x9f, 0xe8, 0x59, 0xc0, 0xad, 0x63, 0xfd, 0xe4, 0x94, 0xf5, 0x03, 0x9e, 0x0e, 0x82, 0x32, 0x61, 0x2b, 0xd5},
+    {0x36, 0xd5, 0x69, 0x07, 0xe2, 0xec, 0x74, 0x5d, 0xb6, 0xe5, 0x4f, 0x0b, 0x2e, 0x1b, 0x23, 0x00, 0xab, 0xcb, 0x42, 0x2e, 0x71, 0x2d, 0xa5, 0x88, 0xa4, 0x0d, 0x3f, 0x1e, 0xbb, 0xbe, 0x02, 0xf6},
+    {0x34, 0xdb, 0x6e, 0xe4, 0xd0, 0x60, 0x8e, 0x5f, 0x78, 0x36, 0x50, 0x49, 0x5a, 0x3b, 0x2f, 0x52, 0x73, 0xc5, 0x13, 0x4e, 0x52, 0x84, 0xe4, 0xfd, 0xf9, 0x66, 0x27, 0xbb, 0x16, 0xe3, 0x1e, 0x6b},
+    {0x8e, 0x76, 0x59, 0xfb, 0x45, 0xa3, 0x78, 0x7d, 0x67, 0x4a, 0xe8, 0x67, 0x31, 0xfa, 0xa2, 0x53, 0x8e, 0xc0, 0xfd, 0xf4, 0x42, 0xab, 0x26, 0xe9, 0xc7, 0x91, 0xfa, 0xda, 0x08, 0x94, 0x67, 0xe9},
+    {0x30, 0x06, 0xcf, 0x19, 0x8b, 0x24, 0xf3, 0x1b, 0xb4, 0xc7, 0xe6, 0x34, 0x60, 0x00, 0xab, 0xc7, 0x01, 0xe8, 0x27, 0xcf, 0xbb, 0x5d, 0xf5, 0x2d, 0xcf, 0xa4, 0x2e, 0x9c, 0xa9, 0xff, 0x08, 0x02},
+    {0xf5, 0xfd, 0x40, 0x3c, 0xb6, 0xe8, 0xbe, 0x21, 0x47, 0x2e, 0x37, 0x7f, 0xfd, 0x80, 0x5a, 0x8c, 0x60, 0x83, 0xea, 0x48, 0x03, 0xb8, 0x48, 0x53, 0x89, 0xcc, 0x3e, 0xbc, 0x21, 0x5f, 0x00, 0x2a},
+    {0x37, 0x31, 0xb2, 0x60, 0xeb, 0x3f, 0x94, 0x82, 0xe4, 0x5f, 0x1c, 0x3f, 0x3b, 0x9d, 0xcf, 0x83, 0x4b, 0x75, 0xe6, 0xee, 0xf8, 0xc4, 0x0f, 0x46, 0x1e, 0xa2, 0x7e, 0x8b, 0x6e, 0xd9, 0x47, 0x3d},
+    {0x9f, 0x9d, 0xab, 0x09, 0xc3, 0xf5, 0xe4, 0x28, 0x55, 0xc2, 0xde, 0x97, 0x1b, 0x65, 0x93, 0x28, 0xa2, 0xdb, 0xc4, 0x54, 0x84, 0x5f, 0x39, 0x6f, 0xfc, 0x05, 0x3f, 0x0b, 0xb1, 0x92, 0xf8, 0xc3},
+    {0x5e, 0x05, 0x5d, 0x25, 0xf8, 0x5f, 0xdb, 0x98, 0xf2, 0x73, 0xe4, 0xaf, 0xe0, 0x84, 0x64, 0xc0, 0x03, 0xb7, 0x0f, 0x1e, 0xf0, 0x67, 0x7b, 0xb5, 0xe2, 0x57, 0x06, 0x40, 0x0b, 0xe6, 0x20, 0xa5},
+    {0x86, 0x8b, 0xcf, 0x36, 0x79, 0xcb, 0x6b, 0x50, 0x0b, 0x94, 0x41, 0x8c, 0x0b, 0x89, 0x25, 0xf9, 0x86, 0x55, 0x30, 0x30, 0x3a, 0xe4, 0xe4, 0xb2, 0x62, 0x59, 0x18, 0x65, 0x66, 0x6a, 0x45, 0x90},
+    {0xb3, 0xdb, 0x6b, 0xd3, 0x89, 0x7a, 0xfb, 0xd1, 0xdf, 0x3f, 0x96, 0x44, 0xab, 0x21, 0xc8, 0x05, 0x0e, 0x1f, 0x00, 0x38, 0xa5, 0x2f, 0x7c, 0xa9, 0x5a, 0xc0, 0xc3, 0xde, 0x75, 0x58, 0xcb, 0x7a},
+    {0x81, 0x19, 0xb3, 0xa0, 0x59, 0xff, 0x2c, 0xac, 0x48, 0x3e, 0x69, 0xbc, 0xd4, 0x1d, 0x6d, 0x27, 0x14, 0x94, 0x47, 0x91, 0x42, 0x88, 0xbb, 0xea, 0xee, 0x34, 0x13, 0xe6, 0xdc, 0xc6, 0xd1, 0xeb},
+    {0x10, 0xfc, 0x58, 0xf3, 0x5f, 0xc7, 0xfe, 0x7a, 0xe8, 0x75, 0x52, 0x4b, 0xb5, 0x85, 0x00, 0x03, 0x00, 0x5b, 0x7f, 0x97, 0x8c, 0x0c, 0x65, 0xe2, 0xa9, 0x65, 0x46, 0x4b, 0x6d, 0x00, 0x81, 0x9c},
+    {0x5a, 0xcd, 0x94, 0xeb, 0x3c, 0x57, 0x83, 0x79, 0xc1, 0xea, 0x58, 0xa3, 0x43, 0xec, 0x4f, 0xcf, 0xf9, 0x62, 0x77, 0x6f, 0xe3, 0x55, 0x21, 0xe4, 0x75, 0xa0, 0xe0, 0x6d, 0x88, 0x7b, 0x2d, 0xb9},
+    {0x33, 0xda, 0xf3, 0xa2, 0x14, 0xd6, 0xe0, 0xd4, 0x2d, 0x23, 0x00, 0xa7, 0xb4, 0x4b, 0x39, 0x29, 0x0d, 0xb8, 0x98, 0x9b, 0x42, 0x79, 0x74, 0xcd, 0x86, 0x5d, 0xb0, 0x11, 0x05, 0x5a, 0x29, 0x01},
+    {0xcf, 0xc6, 0x57, 0x2f, 0x29, 0xaf, 0xd1, 0x64, 0xa4, 0x94, 0xe6, 0x4e, 0x6f, 0x1a, 0xeb, 0x82, 0x0c, 0x3e, 0x7d, 0xa3, 0x55, 0x14, 0x4e, 0x51, 0x24, 0xa3, 0x91, 0xd0, 0x6e, 0x9f, 0x95, 0xea},
+    {0xd5, 0x31, 0x2a, 0x4b, 0x0e, 0xf6, 0x15, 0xa3, 0x31, 0xf6, 0x35, 0x2c, 0x2e, 0xd2, 0x1d, 0xac, 0x9e, 0x7c, 0x36, 0x39, 0x8b, 0x93, 0x9a, 0xec, 0x90, 0x1c, 0x25, 0x7f, 0x6c, 0xbc, 0x9e, 0x8e},
+    {0x55, 0x1d, 0x67, 0xfe, 0xfc, 0x7b, 0x5b, 0x9f, 0x9f, 0xdb, 0xf6, 0xaf, 0x57, 0xc9, 0x6c, 0x8a, 0x74, 0xd7, 0xe4, 0x5a, 0x00, 0x20, 0x78, 0xa7, 0xb5, 0xba, 0x45, 0xc6, 0xfd, 0xe9, 0x3e, 0x33},
+    {0xd5, 0x0a, 0xc7, 0xbd, 0x5c, 0xa5, 0x93, 0xc6, 0x56, 0x92, 0x8f, 0x38, 0x42, 0x80, 0x17, 0xfc, 0x7b, 0xa5, 0x02, 0x85, 0x4c, 0x43, 0xd8, 0x41, 0x49, 0x50, 0xe9, 0x6e, 0xcb, 0x40, 0x5d, 0xc3},
+    {0x07, 0x73, 0xe1, 0x8e, 0xa1, 0xbe, 0x44, 0xfe, 0x1a, 0x97, 0xe2, 0x39, 0x57, 0x3c, 0xfa, 0xe3, 0xe4, 0xe9, 0x5e, 0xf9, 0xaa, 0x9f, 0xaa, 0xbe, 0xac, 0x12, 0x74, 0xd3, 0xad, 0x26, 0x16, 0x04},
+    {0xe9, 0xaf, 0x0e, 0x7c, 0xa8, 0x93, 0x30, 0xd2, 0xb8, 0x61, 0x5d, 0x1b, 0x41, 0x37, 0xca, 0x61, 0x7e, 0x21, 0x29, 0x7f, 0x2f, 0x0d, 0xed, 0x8e, 0x31, 0xb7, 0xd2, 0xea, 0xd8, 0x71, 0x46, 0x60},
+    {0x7b, 0x12, 0x45, 0x83, 0x09, 0x7f, 0x10, 0x29, 0xa0, 0xc7, 0x41, 0x91, 0xfe, 0x73, 0x78, 0xc9, 0x10, 0x5a, 0xcc, 0x70, 0x66, 0x95, 0xed, 0x14, 0x93, 0xbb, 0x76, 0x03, 0x42, 0x26, 0xa5, 0x7b},
+    {0xec, 0x40, 0x05, 0x7b, 0x99, 0x54, 0x76, 0x65, 0x0b, 0x3d, 0xb9, 0x8e, 0x9d, 0xb7, 0x57, 0x38, 0xa8, 0xcd, 0x2f, 0x94, 0xd8, 0x63, 0xb9, 0x06, 0x15, 0x0c, 0x56, 0xaa, 0xc1, 0x9c, 0xaa, 0x6b},
+    {0x01, 0xd9, 0xff, 0x72, 0x9e, 0xfd, 0x39, 0xd8, 0x37, 0x84, 0xc0, 0xfe, 0x59, 0xc4, 0xae, 0x81, 0xa6, 0x70, 0x34, 0xcb, 0x53, 0xc9, 0x43, 0xfb, 0x81, 0x8b, 0x9d, 0x8a, 0xe7, 0xfc, 0x33, 0xe5},
+    {0x00, 0xdf, 0xb3, 0xc6, 0x96, 0x32, 0x8c, 0x76, 0x42, 0x45, 0x19, 0xa7, 0xbe, 0xfe, 0x8e, 0x0f, 0x6c, 0x76, 0xf9, 0x47, 0xb5, 0x27, 0x67, 0x91, 0x6d, 0x24, 0x82, 0x3f, 0x73, 0x5b, 0xaf, 0x2e},
+    {0x46, 0x1b, 0x79, 0x9b, 0x4d, 0x9c, 0xee, 0xa8, 0xd5, 0x80, 0xdc, 0xb7, 0x6d, 0x11, 0x15, 0x0d, 0x53, 0x5e, 0x16, 0x39, 0xd1, 0x60, 0x03, 0xc3, 0xfb, 0x7e, 0x9d, 0x1f, 0xd1, 0x30, 0x83, 0xa8},
+    {0xee, 0x03, 0x03, 0x94, 0x79, 0xe5, 0x22, 0x8f, 0xdc, 0x55, 0x1c, 0xbd, 0xe7, 0x07, 0x9d, 0x34, 0x12, 0xea, 0x18, 0x6a, 0x51, 0x7c, 0xcc, 0x63, 0xe4, 0x6e, 0x9f, 0xcc, 0xe4, 0xfe, 0x3a, 0x6c},
+    {0xa8, 0xcf, 0xb5, 0x43, 0x52, 0x4e, 0x7f, 0x02, 0xb9, 0xf0, 0x45, 0xac, 0xd5, 0x43, 0xc2, 0x1c, 0x37, 0x3b, 0x4c, 0x9b, 0x98, 0xac, 0x20, 0xce, 0xc4, 0x17, 0xa6, 0xdd, 0xb5, 0x74, 0x4e, 0x94},
+    {0x93, 0x2b, 0x79, 0x4b, 0xf8, 0x9c, 0x6e, 0xda, 0xf5, 0xd0, 0x65, 0x0c, 0x7c, 0x4b, 0xad, 0x92, 0x42, 0xb2, 0x56, 0x26, 0xe3, 0x7e, 0xad, 0x5a, 0xa7, 0x5e, 0xc8, 0xc6, 0x4e, 0x09, 0xdd, 0x4f},
+    {0x16, 0xb1, 0x0c, 0x77, 0x9c, 0xe5, 0xcf, 0xef, 0x59, 0xc7, 0x71, 0x0d, 0x2e, 0x68, 0x44, 0x1e, 0xa6, 0xfa, 0xcb, 0x68, 0xe9, 0xb5, 0xf7, 0xd5, 0x33, 0xae, 0x0b, 0xb7, 0x8e, 0x28, 0xbf, 0x57},
+    {0x0f, 0x77, 0xc7, 0x67, 0x43, 0xe7, 0x39, 0x6f, 0x99, 0x10, 0x13, 0x9f, 0x49, 0x37, 0xd8, 0x37, 0xae, 0x54, 0xe2, 0x10, 0x38, 0xac, 0x5c, 0x0b, 0x3f, 0xd6, 0xef, 0x17, 0x1a, 0x28, 0xa7, 0xe4},
+    {0xd7, 0xe5, 0x74, 0xb7, 0xb9, 0x52, 0xf2, 0x93, 0xe8, 0x0d, 0xde, 0x90, 0x5e, 0xb5, 0x09, 0x37, 0x3f, 0x3f, 0x6c, 0xd1, 0x09, 0xa0, 0x22, 0x08, 0xb3, 0xc1, 0xe9, 0x24, 0x08, 0x0a, 0x20, 0xca},
+    {0x45, 0x66, 0x6f, 0x8c, 0x38, 0x1e, 0x3d, 0xa6, 0x75, 0x56, 0x3f, 0xf8, 0xba, 0x23, 0xf8, 0x3b, 0xfa, 0xc3, 0x0c, 0x34, 0xab, 0xdd, 0xe6, 0xe5, 0xc0, 0x97, 0x5e, 0xf9, 0xfd, 0x70, 0x0c, 0xb9},
+    {0xb2, 0x46, 0x12, 0xe4, 0x54, 0x60, 0x7e, 0xb1, 0xab, 0xa4, 0x47, 0xf8, 0x16, 0xd1, 0xa4, 0x55, 0x1e, 0xf9, 0x5f, 0xa7, 0x24, 0x7f, 0xb7, 0xc1, 0xf5, 0x03, 0x02, 0x0a, 0x71, 0x77, 0xf0, 0xdd},
+    {0x7e, 0x20, 0x88, 0x61, 0x85, 0x6d, 0xa4, 0x2c, 0x8b, 0xb4, 0x6a, 0x75, 0x67, 0xf8, 0x12, 0x13, 0x62, 0xd9, 0xfb, 0x24, 0x96, 0xf1, 0x31, 0xa4, 0xaa, 0x90, 0x17, 0xcf, 0x36, 0x6c, 0xdf, 0xce},
+    {0x5b, 0x64, 0x6b, 0xff, 0x6a, 0xd1, 0x10, 0x01, 0x65, 0x03, 0x7a, 0x05, 0x56, 0x01, 0xea, 0x02, 0x35, 0x8c, 0x0f, 0x41, 0x05, 0x0f, 0x9d, 0xfe, 0x3c, 0x95, 0xdc, 0xcb, 0xd3, 0x08, 0x7b, 0xe0},
+    {0x74, 0x6d, 0x1d, 0xcc, 0xfe, 0xd2, 0xf0, 0xff, 0x1e, 0x13, 0xc5, 0x1e, 0x2d, 0x50, 0xd5, 0x32, 0x43, 0x75, 0xfb, 0xd5, 0xbf, 0x7c, 0xa8, 0x2a, 0x89, 0x31, 0x82, 0x8d, 0x80, 0x1d, 0x43, 0xab},
+    {0xcb, 0x98, 0x11, 0x0d, 0x4a, 0x6b, 0xb9, 0x7d, 0x22, 0xfe, 0xad, 0xbc, 0x6c, 0x0d, 0x89, 0x30, 0xc5, 0xf8, 0xfc, 0x50, 0x8b, 0x2f, 0xc5, 0xb3, 0x53, 0x28, 0xd2, 0x6b, 0x88, 0xdb, 0x19, 0xae},
+    {0x60, 0xb6, 0x26, 0xa0, 0x33, 0xb5, 0x5f, 0x27, 0xd7, 0x67, 0x6c, 0x40, 0x95, 0xea, 0xba, 0xbc, 0x7a, 0x2c, 0x7e, 0xde, 0x26, 0x24, 0xb4, 0x72, 0xe9, 0x7f, 0x64, 0xf9, 0x6b, 0x8c, 0xfc, 0x0e},
+    {0xe5, 0xb5, 0x2b, 0xc9, 0x27, 0x46, 0x8d, 0xf7, 0x18, 0x93, 0xeb, 0x81, 0x97, 0xef, 0x82, 0x0c, 0xf7, 0x6c, 0xb0, 0xaa, 0xf6, 0xe8, 0xe4, 0xfe, 0x93, 0xad, 0x62, 0xd8, 0x03, 0x98, 0x31, 0x04},
+    {0x05, 0x65, 0x41, 0xae, 0x5d, 0xa9, 0x96, 0x1b, 0xe2, 0xb0, 0xa5, 0xe8, 0x95, 0xe5, 0xc5, 0xba, 0x15, 0x3c, 0xbb, 0x62, 0xdd, 0x56, 0x1a, 0x42, 0x7b, 0xad, 0x0f, 0xfd, 0x41, 0x92, 0x31, 0x99},
+    {0xf8, 0xfe, 0xf0, 0x5a, 0x3f, 0xa5, 0xc9, 0xf3, 0xeb, 0xa4, 0x16, 0x38, 0xb2, 0x47, 0xb7, 0x11, 0xa9, 0x9f, 0x96, 0x0f, 0xe7, 0x3a, 0xa2, 0xf9, 0x01, 0x36, 0xae, 0xb2, 0x03, 0x29, 0xb8, 0x88}};
+
+    //Debug printing for the above types
+    //Actually use DP(value) and #define DBG
+    void dp(key a);
+    void dp(bool a);
+    void dp(const char * a, int l);
+    void dp(keyV a);
+    void dp(keyM a);
+    void dp(xmr_amount vali);
+    void dp(int vali);
+    void dp(bits amountb);
+    void dp(const char * st);
+
+    //various conversions
+
+    //uint long long to 32 byte key
+    void d2h(key & amounth, xmr_amount val);
+    key d2h(xmr_amount val);
+    //uint long long to int[64]
+    void d2b(bits  amountb, xmr_amount val);
+    //32 byte key to uint long long
+    // if the key holds a value > 2^64
+    // then the value in the first 8 bytes is returned
+    xmr_amount h2d(const key &test);
+    //32 byte key to int[64]
+    void h2b(bits  amountb2, key & test);
+    //int[64] to 32 byte key
+    void b2h(key  & amountdh, bits amountb2);
+    //int[64] to uint long long
+    xmr_amount b2d(bits amountb);
+
+    static inline const rct::key pk2rct(const crypto::public_key &pk) { return (const rct::key&)pk; }
+    static inline const rct::key sk2rct(const crypto::secret_key &sk) { return (const rct::key&)sk; }
+    static inline const rct::key ki2rct(const crypto::key_image &ki) { return (const rct::key&)ki; }
+    static inline const rct::key hash2rct(const crypto::hash &h) { return (const rct::key&)h; }
+    static inline const crypto::public_key rct2pk(const rct::key &k) { return (const crypto::public_key&)k; }
+    static inline const crypto::secret_key rct2sk(const rct::key &k) { return (const crypto::secret_key&)k; }
+    static inline const crypto::key_image rct2ki(const rct::key &k) { return (const crypto::key_image&)k; }
+    static inline const crypto::hash rct2hash(const rct::key &k) { return (const crypto::hash&)k; }
+    static inline bool operator==(const rct::key &k0, const crypto::public_key &k1) { return !memcmp(&k0, &k1, 32); }
+    static inline bool operator!=(const rct::key &k0, const crypto::public_key &k1) { return memcmp(&k0, &k1, 32); }
+}
+
+
+namespace cryptonote {
+    static inline bool operator==(const crypto::public_key &k0, const rct::key &k1) { return !memcmp(&k0, &k1, 32); }
+    static inline bool operator!=(const crypto::public_key &k0, const rct::key &k1) { return memcmp(&k0, &k1, 32); }
+    static inline bool operator==(const crypto::secret_key &k0, const rct::key &k1) { return !memcmp(&k0, &k1, 32); }
+    static inline bool operator!=(const crypto::secret_key &k0, const rct::key &k1) { return memcmp(&k0, &k1, 32); }
+}
+
+template<typename T> std::ostream &print256(std::ostream &o, const T &v);
+inline std::ostream &operator <<(std::ostream &o, const rct::key &v) { return print256(o, v); }
+
+
+BLOB_SERIALIZER(rct::key);
+BLOB_SERIALIZER(rct::key64);
+BLOB_SERIALIZER(rct::ctkey);
+BLOB_SERIALIZER(rct::asnlSig);
+
+VARIANT_TAG(debug_archive, rct::key, "rct::key");
+VARIANT_TAG(debug_archive, rct::key64, "rct::key64");
+VARIANT_TAG(debug_archive, rct::keyV, "rct::keyV");
+VARIANT_TAG(debug_archive, rct::keyM, "rct::keyM");
+VARIANT_TAG(debug_archive, rct::ctkey, "rct::ctkey");
+VARIANT_TAG(debug_archive, rct::ctkeyV, "rct::ctkeyV");
+VARIANT_TAG(debug_archive, rct::ctkeyM, "rct::ctkeyM");
+VARIANT_TAG(debug_archive, rct::ecdhTuple, "rct::ecdhTuple");
+VARIANT_TAG(debug_archive, rct::mgSig, "rct::mgSig");
+VARIANT_TAG(debug_archive, rct::rangeSig, "rct::rangeSig");
+VARIANT_TAG(debug_archive, rct::asnlSig, "rct::asnlSig");
+VARIANT_TAG(debug_archive, rct::rctSig, "rct::rctSig");
+
+VARIANT_TAG(binary_archive, rct::key, 0x90);
+VARIANT_TAG(binary_archive, rct::key64, 0x91);
+VARIANT_TAG(binary_archive, rct::keyV, 0x92);
+VARIANT_TAG(binary_archive, rct::keyM, 0x93);
+VARIANT_TAG(binary_archive, rct::ctkey, 0x94);
+VARIANT_TAG(binary_archive, rct::ctkeyV, 0x95);
+VARIANT_TAG(binary_archive, rct::ctkeyM, 0x96);
+VARIANT_TAG(binary_archive, rct::ecdhTuple, 0x97);
+VARIANT_TAG(binary_archive, rct::mgSig, 0x98);
+VARIANT_TAG(binary_archive, rct::rangeSig, 0x99);
+VARIANT_TAG(binary_archive, rct::asnlSig, 0x9a);
+VARIANT_TAG(binary_archive, rct::rctSig, 0x9b);
+
+VARIANT_TAG(json_archive, rct::key, "rct_key");
+VARIANT_TAG(json_archive, rct::key64, "rct_key64");
+VARIANT_TAG(json_archive, rct::keyV, "rct_keyV");
+VARIANT_TAG(json_archive, rct::keyM, "rct_keyM");
+VARIANT_TAG(json_archive, rct::ctkey, "rct_ctkey");
+VARIANT_TAG(json_archive, rct::ctkeyV, "rct_ctkeyV");
+VARIANT_TAG(json_archive, rct::ctkeyM, "rct_ctkeyM");
+VARIANT_TAG(json_archive, rct::ecdhTuple, "rct_ecdhTuple");
+VARIANT_TAG(json_archive, rct::mgSig, "rct_mgSig");
+VARIANT_TAG(json_archive, rct::rangeSig, "rct_rangeSig");
+VARIANT_TAG(json_archive, rct::asnlSig, "rct_asnlSig");
+VARIANT_TAG(json_archive, rct::rctSig, "rct_rctSig");
+
+#endif  /* RCTTYPES_H */

src/rpc/core_rpc_server.cpp

@@ -159,9 +159,25 @@ namespace cryptonote
     {
       res.blocks.resize(res.blocks.size()+1);
       res.blocks.back().block = block_to_blob(b.first);
+      res.output_indices.push_back(COMMAND_RPC_GET_BLOCKS_FAST::block_output_indices());
+      res.output_indices.back().indices.push_back(COMMAND_RPC_GET_BLOCKS_FAST::tx_output_indices());
+      bool r = m_core.get_tx_outputs_gindexs(get_transaction_hash(b.first.miner_tx), res.output_indices.back().indices.back().indices);
+      if (!r)
+      {
+        res.status = "Failed";
+        return false;
+      }
+      size_t txidx = 0;
       BOOST_FOREACH(auto& t, b.second)
       {
         res.blocks.back().txs.push_back(tx_to_blob(t));
+        res.output_indices.back().indices.push_back(COMMAND_RPC_GET_BLOCKS_FAST::tx_output_indices());
+        bool r = m_core.get_tx_outputs_gindexs(b.first.tx_hashes[txidx++], res.output_indices.back().indices.back().indices);
+        if (!r)
+        {
+          res.status = "Failed";
+          return false;
+        }
       }
     }
 
@@ -250,6 +266,30 @@ namespace cryptonote
     return true;
   }
   //------------------------------------------------------------------------------------------------------------------------------
+  bool core_rpc_server::on_get_random_rct_outs(const COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::request& req, COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::response& res)
+  {
+    CHECK_CORE_BUSY();
+    res.status = "Failed";
+    if(!m_core.get_random_rct_outs(req, res))
+    {
+      return true;
+    }
+
+    res.status = CORE_RPC_STATUS_OK;
+    std::stringstream ss;
+    typedef COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::out_entry out_entry;
+    CHECK_AND_ASSERT_MES(res.outs.size(), true, "internal error: res.outs.size() is empty");
+    std::for_each(res.outs.begin(), res.outs.end(), [&](out_entry& oe)
+      {
+        ss << oe.global_amount_index << " ";
+      });
+    ss << ENDL;
+    std::string s = ss.str();
+    LOG_PRINT_L2("COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS: " << ENDL << s);
+    res.status = CORE_RPC_STATUS_OK;
+    return true;
+  }
+  //------------------------------------------------------------------------------------------------------------------------------
   bool core_rpc_server::on_get_indexes(const COMMAND_RPC_GET_TX_GLOBAL_OUTPUTS_INDEXES::request& req, COMMAND_RPC_GET_TX_GLOBAL_OUTPUTS_INDEXES::response& res)
   {
     CHECK_CORE_BUSY();
@@ -459,6 +499,8 @@ namespace cryptonote
         res.reason = "overspend";
       if ((res.fee_too_low = tvc.m_fee_too_low))
         res.reason = "fee too low";
+      if ((res.not_rct = tvc.m_not_rct))
+        res.reason = "tx is not ringct";
       return true;
     }
 
@@ -1023,7 +1065,6 @@ namespace cryptonote
       return false;
     }
 
-#if BLOCKCHAIN_DB == DB_LMDB
     const Blockchain &blockchain = m_core.get_blockchain_storage();
     uint8_t version = req.version > 0 ? req.version : blockchain.get_next_hard_fork_version();
     res.version = blockchain.get_current_hard_fork_version();
@@ -1031,11 +1072,6 @@ namespace cryptonote
     res.state = blockchain.get_hard_fork_state();
     res.status = CORE_RPC_STATUS_OK;
     return true;
-#else
-    error_resp.code = CORE_RPC_ERROR_CODE_UNSUPPORTED_RPC;
-    error_resp.message = "Hard fork inoperative in memory mode.";
-    return false;
-#endif
   }
   //------------------------------------------------------------------------------------------------------------------------------
   bool core_rpc_server::on_get_bans(const COMMAND_RPC_GETBANS::request& req, COMMAND_RPC_GETBANS::response& res, epee::json_rpc::error& error_resp)

src/rpc/core_rpc_server.h

@@ -79,6 +79,7 @@ namespace cryptonote
       MAP_URI_AUTO_BIN2("/get_o_indexes.bin", on_get_indexes, COMMAND_RPC_GET_TX_GLOBAL_OUTPUTS_INDEXES)      
       MAP_URI_AUTO_BIN2("/getrandom_outs.bin", on_get_random_outs, COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS)      
       MAP_URI_AUTO_BIN2("/get_outs.bin", on_get_outs, COMMAND_RPC_GET_OUTPUTS)      
+      MAP_URI_AUTO_BIN2("/getrandom_rctouts.bin", on_get_random_rct_outs, COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS)
       MAP_URI_AUTO_JON2("/gettransactions", on_get_transactions, COMMAND_RPC_GET_TRANSACTIONS)
       MAP_URI_AUTO_JON2("/is_key_image_spent", on_is_key_image_spent, COMMAND_RPC_IS_KEY_IMAGE_SPENT)
       MAP_URI_AUTO_JON2("/sendrawtransaction", on_send_raw_tx, COMMAND_RPC_SEND_RAW_TX)
@@ -128,6 +129,7 @@ namespace cryptonote
     bool on_mining_status(const COMMAND_RPC_MINING_STATUS::request& req, COMMAND_RPC_MINING_STATUS::response& res);
     bool on_get_random_outs(const COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::request& req, COMMAND_RPC_GET_RANDOM_OUTPUTS_FOR_AMOUNTS::response& res);        
     bool on_get_outs(const COMMAND_RPC_GET_OUTPUTS::request& req, COMMAND_RPC_GET_OUTPUTS::response& res);        
+    bool on_get_random_rct_outs(const COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::request& req, COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS::response& res);
     bool on_get_info(const COMMAND_RPC_GET_INFO::request& req, COMMAND_RPC_GET_INFO::response& res);        
     bool on_save_bc(const COMMAND_RPC_SAVE_BC::request& req, COMMAND_RPC_SAVE_BC::response& res);
     bool on_get_peer_list(const COMMAND_RPC_GET_PEER_LIST::request& req, COMMAND_RPC_GET_PEER_LIST::response& res);

src/rpc/core_rpc_server_commands_defs.h

@@ -41,7 +41,7 @@ namespace cryptonote
 #define CORE_RPC_STATUS_BUSY   "BUSY"
 #define CORE_RPC_STATUS_NOT_MINING "NOT MINING"
 
-#define CORE_RPC_VERSION 2
+#define CORE_RPC_VERSION 3
 
   struct COMMAND_RPC_GET_HEIGHT
   {
@@ -76,18 +76,38 @@ namespace cryptonote
       END_KV_SERIALIZE_MAP()
     };
 
+    struct tx_output_indices
+    {
+      std::vector<uint64_t> indices;
+
+      BEGIN_KV_SERIALIZE_MAP()
+        KV_SERIALIZE(indices)
+      END_KV_SERIALIZE_MAP()
+    };
+
+    struct block_output_indices
+    {
+      std::vector<tx_output_indices> indices;
+
+      BEGIN_KV_SERIALIZE_MAP()
+        KV_SERIALIZE(indices)
+      END_KV_SERIALIZE_MAP()
+    };
+
     struct response
     {
       std::list<block_complete_entry> blocks;
       uint64_t    start_height;
       uint64_t    current_height;
       std::string status;
+      std::vector<block_output_indices> output_indices;
 
       BEGIN_KV_SERIALIZE_MAP()
         KV_SERIALIZE(blocks)
         KV_SERIALIZE(start_height)
         KV_SERIALIZE(current_height)
         KV_SERIALIZE(status)
+        KV_SERIALIZE(output_indices)
       END_KV_SERIALIZE_MAP()
     };
   };
@@ -296,10 +316,12 @@ namespace cryptonote
     struct outkey
     {
       crypto::public_key key;
+      rct::key mask;
       bool unlocked;
 
       BEGIN_KV_SERIALIZE_MAP()
         KV_SERIALIZE_VAL_POD_AS_BLOB(key)
+        KV_SERIALIZE_VAL_POD_AS_BLOB(mask)
         KV_SERIALIZE(unlocked)
       END_KV_SERIALIZE_MAP()
     };
@@ -315,6 +337,37 @@ namespace cryptonote
       END_KV_SERIALIZE_MAP()
     };
   };
+
+  struct COMMAND_RPC_GET_RANDOM_RCT_OUTPUTS
+  {
+    struct request
+    {
+      uint64_t outs_count;
+      BEGIN_KV_SERIALIZE_MAP()
+        KV_SERIALIZE(outs_count)
+      END_KV_SERIALIZE_MAP()
+    };
+
+#pragma pack (push, 1)
+    struct out_entry
+    {
+      uint64_t amount;
+      uint64_t global_amount_index;
+      crypto::public_key out_key;
+      rct::key commitment;
+    };
+#pragma pack(pop)
+
+    struct response
+    {
+      std::list<out_entry> outs;
+      std::string status;
+      BEGIN_KV_SERIALIZE_MAP()
+        KV_SERIALIZE_CONTAINER_POD_AS_BLOB(outs)
+        KV_SERIALIZE(status)
+      END_KV_SERIALIZE_MAP()
+    };
+  };
   //-----------------------------------------------
   struct COMMAND_RPC_SEND_RAW_TX
   {
@@ -345,6 +398,7 @@ namespace cryptonote
       bool too_big;
       bool overspend;
       bool fee_too_low;
+      bool not_rct;
 
       BEGIN_KV_SERIALIZE_MAP()
         KV_SERIALIZE(status)
@@ -357,6 +411,7 @@ namespace cryptonote
         KV_SERIALIZE(too_big)
         KV_SERIALIZE(overspend)
         KV_SERIALIZE(fee_too_low)
+        KV_SERIALIZE(not_rct)
       END_KV_SERIALIZE_MAP()
     };
   };

src/serialization/crypto.h

@@ -92,3 +92,4 @@ VARIANT_TAG(debug_archive, crypto::secret_key, "secret_key");
 VARIANT_TAG(debug_archive, crypto::key_derivation, "key_derivation");
 VARIANT_TAG(debug_archive, crypto::key_image, "key_image");
 VARIANT_TAG(debug_archive, crypto::signature, "signature");
+

src/serialization/serialization.h

@@ -102,6 +102,12 @@ inline bool do_serialize(Archive &ar, T &v)
 {
   return ::serializer<Archive, T>::serialize(ar, v);
 }
+template <class Archive>
+inline bool do_serialize(Archive &ar, bool &v)
+{
+  ar.serialize_blob(&v, sizeof(v));
+  return true;
+}
 
 // Never used in the code base
 // #ifndef __GNUC__

src/serialization/vector.h

@@ -32,6 +32,11 @@
 
 #include "serialization.h"
 
+template <template <bool> class Archive, class T>
+bool do_serialize(Archive<false> &ar, std::vector<T> &v);
+template <template <bool> class Archive, class T>
+bool do_serialize(Archive<true> &ar, std::vector<T> &v);
+
 namespace serialization
 {
   namespace detail

src/simplewallet/simplewallet.cpp

@@ -58,6 +58,7 @@
 #include "mnemonics/electrum-words.h"
 #include "rapidjson/document.h"
 #include "common/json_util.h"
+#include "ringct/rctSigs.h"
 #include <stdexcept>
 
 #if defined(WIN32)
@@ -99,6 +100,11 @@ typedef cryptonote::simple_wallet sw;
     m_auto_refresh_enabled.store(auto_refresh_enabled, std::memory_order_relaxed); \
   })
 
+enum TransferType {
+  TransferOriginal,
+  TransferNew,
+};
+
 namespace
 {
   const command_line::arg_descriptor<std::string> arg_wallet_file = {"wallet-file", sw::tr("Use wallet <arg>"), ""};
@@ -642,8 +648,8 @@ simple_wallet::simple_wallet()
   m_cmd_binder.set_handler("incoming_transfers", boost::bind(&simple_wallet::show_incoming_transfers, this, _1), tr("incoming_transfers [available|unavailable] - Show incoming transfers, all or filtered by availability"));
   m_cmd_binder.set_handler("payments", boost::bind(&simple_wallet::show_payments, this, _1), tr("payments <PID_1> [<PID_2> ... <PID_N>] - Show payments for given payment ID[s]"));
   m_cmd_binder.set_handler("bc_height", boost::bind(&simple_wallet::show_blockchain_height, this, _1), tr("Show blockchain height"));
-  m_cmd_binder.set_handler("transfer", boost::bind(&simple_wallet::transfer, this, _1), tr("transfer [<mixin_count>] <addr_1> <amount_1> [<addr_2> <amount_2> ... <addr_N> <amount_N>] [payment_id] - Transfer <amount_1>,... <amount_N> to <address_1>,... <address_N>, respectively. <mixin_count> is the number of extra inputs to include for untraceability (from 0 to maximum available)"));
+  m_cmd_binder.set_handler("transfer_original", boost::bind(&simple_wallet::transfer, this, _1), tr("transfer [<mixin_count>] <addr_1> <amount_1> [<addr_2> <amount_2> ... <addr_N> <amount_N>] [payment_id] - Transfer <amount_1>,... <amount_N> to <address_1>,... <address_N>, respectively. <mixin_count> is the number of extra inputs to include for untraceability (from 0 to maximum available)"));
-  m_cmd_binder.set_handler("transfer_new", boost::bind(&simple_wallet::transfer_new, this, _1), tr("Same as transfer, but using a new transaction building algorithm"));
+  m_cmd_binder.set_handler("transfer", boost::bind(&simple_wallet::transfer_new, this, _1), tr("Same as transfer_original, but using a new transaction building algorithm"));
   m_cmd_binder.set_handler("sweep_unmixable", boost::bind(&simple_wallet::sweep_unmixable, this, _1), tr("Send all unmixable outputs to yourself with mixin 0"));
   m_cmd_binder.set_handler("sweep_all", boost::bind(&simple_wallet::sweep_all, this, _1), tr("Send all unlocked balance an address"));
   m_cmd_binder.set_handler("set_log", boost::bind(&simple_wallet::set_log, this, _1), tr("set_log <level> - Change current log detail level, <0-4>"));
@@ -973,22 +979,22 @@ bool simple_wallet::generate_from_json(const boost::program_options::variables_m
     return false;
   }
 
-  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, version, unsigned, Uint, true);
+  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, version, unsigned, Uint, true, 0);
   const int current_version = 1;
   if (field_version > current_version) {
     fail_msg_writer() << boost::format(tr("Version %u too new, we can only grok up to %u")) % field_version % current_version;
     return false;
   }
 
-  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, filename, std::string, String, true);
+  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, filename, std::string, String, true, std::string());
 
-  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, scan_from_height, uint64_t, Uint64, false);
+  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, scan_from_height, uint64_t, Uint64, false, 0);
   bool recover = field_scan_from_height_found;
 
-  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, password, std::string, String, false);
+  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, password, std::string, String, false, std::string());
   password = field_password;
 
-  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, viewkey, std::string, String, false);
+  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, viewkey, std::string, String, false, std::string());
   crypto::secret_key viewkey;
   if (field_viewkey_found)
   {
@@ -1006,7 +1012,7 @@ bool simple_wallet::generate_from_json(const boost::program_options::variables_m
     }
   }
 
-  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, spendkey, std::string, String, false);
+  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, spendkey, std::string, String, false, std::string());
   crypto::secret_key spendkey;
   if (field_spendkey_found)
   {
@@ -1024,7 +1030,7 @@ bool simple_wallet::generate_from_json(const boost::program_options::variables_m
     }
   }
 
-  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, seed, std::string, String, false);
+  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, seed, std::string, String, false, std::string());
   std::string old_language;
   if (field_seed_found)
   {
@@ -1037,7 +1043,7 @@ bool simple_wallet::generate_from_json(const boost::program_options::variables_m
     m_restore_deterministic_wallet = true;
   }
 
-  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, address, std::string, String, false);
+  GET_FIELD_FROM_JSON_RETURN_ON_ERROR(json, address, std::string, String, false, std::string());
 
   // compatibility checks
   if (!field_seed_found && !field_viewkey_found)
@@ -1888,24 +1894,24 @@ void simple_wallet::on_new_block(uint64_t height, const cryptonote::block& block
     m_refresh_progress_reporter.update(height, false);
 }
 //----------------------------------------------------------------------------------------------------
-void simple_wallet::on_money_received(uint64_t height, const cryptonote::transaction& tx, size_t out_index)
+void simple_wallet::on_money_received(uint64_t height, const cryptonote::transaction& tx, uint64_t amount)
 {
   message_writer(epee::log_space::console_color_green, false) << "\r" <<
     tr("Height ") << height << ", " <<
     tr("transaction ") << get_transaction_hash(tx) << ", " <<
-    tr("received ") << print_money(tx.vout[out_index].amount);
+    tr("received ") << print_money(amount);
   if (m_auto_refresh_refreshing)
     m_cmd_binder.print_prompt();
   else
     m_refresh_progress_reporter.update(height, true);
 }
 //----------------------------------------------------------------------------------------------------
-void simple_wallet::on_money_spent(uint64_t height, const cryptonote::transaction& in_tx, size_t out_index, const cryptonote::transaction& spend_tx)
+void simple_wallet::on_money_spent(uint64_t height, const cryptonote::transaction& in_tx, uint64_t amount, const cryptonote::transaction& spend_tx)
 {
   message_writer(epee::log_space::console_color_magenta, false) << "\r" <<
     tr("Height ") << height << ", " <<
     tr("transaction ") << get_transaction_hash(spend_tx) << ", " <<
-    tr("spent ") << print_money(in_tx.vout[out_index].amount);
+    tr("spent ") << print_money(amount);
   if (m_auto_refresh_refreshing)
     m_cmd_binder.print_prompt();
   else
@@ -2052,15 +2058,17 @@ bool simple_wallet::show_incoming_transfers(const std::vector<std::string>& args
     {
       if (!transfers_found)
       {
-        message_writer() << boost::format("%21s%8s%16s%68s") % tr("amount") % tr("spent") % tr("global index") % tr("tx id");
+        message_writer() << boost::format("%21s%8s%12s%8s%16s%68s") % tr("amount") % tr("spent") % tr("unlocked") % tr("ringct") % tr("global index") % tr("tx id");
         transfers_found = true;
       }
       message_writer(td.m_spent ? epee::log_space::console_color_magenta : epee::log_space::console_color_green, false) <<
-        boost::format("%21s%8s%16u%68s") %
+        boost::format("%21s%8s%12s%8s%16u%68s") %
         print_money(td.amount()) %
         (td.m_spent ? tr("T") : tr("F")) %
+        (m_wallet->is_transfer_unlocked(td) ? tr("unlocked") : tr("locked")) %
+        (td.is_rct() ? tr("RingCT") : tr("-")) %
         td.m_global_output_index %
-        get_transaction_hash (td.m_tx);
+        td.m_txid;
     }
   }
 
@@ -2274,7 +2282,7 @@ bool simple_wallet::get_address_from_str(const std::string &str, cryptonote::acc
     return true;
 }
 //----------------------------------------------------------------------------------------------------
-bool simple_wallet::transfer_main(bool new_algorithm, const std::vector<std::string> &args_)
+bool simple_wallet::transfer_main(int transfer_type, const std::vector<std::string> &args_)
 {
   if (!try_connect_to_daemon())
     return true;
@@ -2386,10 +2394,17 @@ bool simple_wallet::transfer_main(bool new_algorithm, const std::vector<std::str
   {
     // figure out what tx will be necessary
     std::vector<tools::wallet2::pending_tx> ptx_vector;
-    if (new_algorithm)
+    switch (transfer_type)
+    {
+      case TransferNew:
         ptx_vector = m_wallet->create_transactions_2(dsts, fake_outs_count, 0 /* unlock_time */, 0 /* unused fee arg*/, extra, m_trusted_daemon);
-    else
+      break;
+      default:
+        LOG_ERROR("Unknown transfer method, using original");
+      case TransferOriginal:
         ptx_vector = m_wallet->create_transactions(dsts, fake_outs_count, 0 /* unlock_time */, 0 /* unused fee arg*/, extra, m_trusted_daemon);
+        break;
+    }
 
     // if more than one tx necessary, prompt user to confirm
     if (m_wallet->always_confirm_transfers() || ptx_vector.size() > 1)
@@ -2531,14 +2546,13 @@ bool simple_wallet::transfer_main(bool new_algorithm, const std::vector<std::str
 //----------------------------------------------------------------------------------------------------
 bool simple_wallet::transfer(const std::vector<std::string> &args_)
 {
-  return transfer_main(false, args_);
+  return transfer_main(TransferOriginal, args_);
 }
 //----------------------------------------------------------------------------------------------------
 bool simple_wallet::transfer_new(const std::vector<std::string> &args_)
 {
-  return transfer_main(true, args_);
+  return transfer_main(TransferNew, args_);
 }
-
 //----------------------------------------------------------------------------------------------------
 bool simple_wallet::sweep_unmixable(const std::vector<std::string> &args_)
 {
@@ -2944,15 +2958,15 @@ bool simple_wallet::get_tx_key(const std::vector<std::string> &args_)
   LOCK_IDLE_SCOPE();
 
   crypto::secret_key tx_key;
-  bool r = m_wallet->get_tx_key(txid, tx_key);
+  std::vector<crypto::secret_key> amount_keys;
-  if (r)
+  if (m_wallet->get_tx_key(txid, tx_key))
   {
-    success_msg_writer() << tr("Tx key: ") << tx_key;
+    success_msg_writer() << tr("Tx key: ") << epee::string_tools::pod_to_hex(tx_key);
     return true;
   }
   else
   {
-    fail_msg_writer() << tr("no tx key found for this txid");
+    fail_msg_writer() << tr("no tx keys found for this txid");
     return true;
   }
 }
@@ -2979,13 +2993,19 @@ bool simple_wallet::check_tx_key(const std::vector<std::string> &args_)
 
   LOCK_IDLE_SCOPE();
 
+  if (local_args[1].size() < 64 || local_args[1].size() % 64)
+  {
+    fail_msg_writer() << tr("failed to parse tx key");
+    return true;
+  }
+  crypto::secret_key tx_key;
   cryptonote::blobdata tx_key_data;
   if(!epee::string_tools::parse_hexstr_to_binbuff(local_args[1], tx_key_data))
   {
     fail_msg_writer() << tr("failed to parse tx key");
     return true;
   }
-  crypto::secret_key tx_key = *reinterpret_cast<const crypto::secret_key*>(tx_key_data.data());
+  tx_key = *reinterpret_cast<const crypto::secret_key*>(tx_key_data.data());
 
   cryptonote::account_public_address address;
   bool has_payment_id;
@@ -3000,14 +3020,14 @@ bool simple_wallet::check_tx_key(const std::vector<std::string> &args_)
   COMMAND_RPC_GET_TRANSACTIONS::response res;
   req.txs_hashes.push_back(epee::string_tools::pod_to_hex(txid));
   if (!net_utils::invoke_http_json_remote_command2(m_daemon_address + "/gettransactions", req, res, m_http_client) ||
-      (res.txs.empty() && res.txs_as_hex.empty()))
+      (res.txs.size() != 1 && res.txs_as_hex.size() != 1))
   {
     fail_msg_writer() << tr("failed to get transaction from daemon");
     return true;
   }
   cryptonote::blobdata tx_data;
   bool ok;
-  if (!res.txs.empty())
+  if (res.txs.size() == 1)
     ok = string_tools::parse_hexstr_to_binbuff(res.txs.front().as_hex, tx_data);
   else
     ok = string_tools::parse_hexstr_to_binbuff(res.txs_as_hex.front(), tx_data);
@@ -3046,13 +3066,49 @@ bool simple_wallet::check_tx_key(const std::vector<std::string> &args_)
       crypto::public_key pubkey;
       derive_public_key(derivation, n, address.m_spend_public_key, pubkey);
       if (pubkey == tx_out_to_key.key)
-        received += tx.vout[n].amount;
-    }
-  }
-  catch(...)
       {
-    LOG_ERROR("unknown error");
+        uint64_t amount;
-    fail_msg_writer() << tr("unknown error");
+        if (tx.version == 1)
+        {
+          amount = tx.vout[n].amount;
+        }
+        else
+        {
+          try
+          {
+            rct::key Ctmp;
+            //rct::key amount_key = rct::hash_to_scalar(rct::scalarmultKey(rct::pk2rct(address.m_view_public_key), rct::sk2rct(tx_key)));
+            crypto::key_derivation derivation;
+            bool r = crypto::generate_key_derivation(address.m_view_public_key, tx_key, derivation);
+            if (!r)
+            {
+              LOG_ERROR("Failed to generate key derivation to decode rct output " << n);
+              amount = 0;
+            }
+            else
+            {
+              crypto::secret_key scalar1;
+              crypto::derivation_to_scalar(derivation, n, scalar1);
+              rct::ecdhTuple ecdh_info = tx.rct_signatures.ecdhInfo[n];
+              rct::ecdhDecode(ecdh_info, rct::sk2rct(scalar1));
+              rct::key C = tx.rct_signatures.outPk[n].mask;
+              rct::addKeys2(Ctmp, ecdh_info.mask, ecdh_info.amount, rct::H);
+              if (rct::equalKeys(C, Ctmp))
+                amount = rct::h2d(ecdh_info.amount);
+              else
+                amount = 0;
+            }
+          }
+          catch (...) { amount = 0; }
+        }
+        received += amount;
+      }
+    }
+  }
+  catch(const std::exception &e)
+  {
+    LOG_ERROR("error: " << e.what());
+    fail_msg_writer() << tr("error: ") << e.what();
     return true;
   }
 
@@ -3064,6 +3120,24 @@ bool simple_wallet::check_tx_key(const std::vector<std::string> &args_)
   {
     fail_msg_writer() << get_account_address_as_str(m_wallet->testnet(), address) << " " << tr("received nothing in txid") << " " << txid;
   }
+  if (res.txs.front().in_pool)
+  {
+    success_msg_writer() << tr("WARNING: this transaction is not yet included in the blockchain!");
+  }
+  else
+  {
+    std::string err;
+    uint64_t bc_height = get_daemon_blockchain_height(err);
+    if (err.empty())
+    {
+      uint64_t confirmations = bc_height - (res.txs.front().block_height + 1);
+      success_msg_writer() << boost::format(tr("This transaction has %u confirmations")) % confirmations;
+    }
+    else
+    {
+      success_msg_writer() << tr("WARNING: failed to determine number of confirmations!");
+    }
+  }
 
   return true;
 }
@@ -3228,9 +3302,8 @@ bool simple_wallet::show_transfers(const std::vector<std::string> &args_)
     m_wallet->get_unconfirmed_payments_out(upayments);
     for (std::list<std::pair<crypto::hash, tools::wallet2::unconfirmed_transfer_details>>::const_iterator i = upayments.begin(); i != upayments.end(); ++i) {
       const tools::wallet2::unconfirmed_transfer_details &pd = i->second;
-      uint64_t amount = 0;
+      uint64_t amount = pd.m_amount_in;
-      cryptonote::get_inputs_money_amount(pd.m_tx, amount);
+      uint64_t fee = amount - pd.m_amount_out;
-      uint64_t fee = amount - get_outs_money_amount(pd.m_tx);
       std::string payment_id = string_tools::pod_to_hex(i->second.m_payment_id);
       if (payment_id.substr(16).find_first_not_of('0') == std::string::npos)
         payment_id = payment_id.substr(0,16);

src/simplewallet/simplewallet.h

@@ -120,7 +120,7 @@ namespace cryptonote
     bool show_incoming_transfers(const std::vector<std::string> &args);
     bool show_payments(const std::vector<std::string> &args);
     bool show_blockchain_height(const std::vector<std::string> &args);
-    bool transfer_main(bool new_algorithm, const std::vector<std::string> &args);
+    bool transfer_main(int transfer_type, const std::vector<std::string> &args);
     bool transfer(const std::vector<std::string> &args);
     bool transfer_new(const std::vector<std::string> &args);
     bool sweep_all(const std::vector<std::string> &args);
@@ -171,8 +171,8 @@ namespace cryptonote
 
     //----------------- i_wallet2_callback ---------------------
     virtual void on_new_block(uint64_t height, const cryptonote::block& block);
-    virtual void on_money_received(uint64_t height, const cryptonote::transaction& tx, size_t out_index);
+    virtual void on_money_received(uint64_t height, const cryptonote::transaction& tx, uint64_t amount);
-    virtual void on_money_spent(uint64_t height, const cryptonote::transaction& in_tx, size_t out_index, const cryptonote::transaction& spend_tx);
+    virtual void on_money_spent(uint64_t height, const cryptonote::transaction& in_tx, uint64_t amount, const cryptonote::transaction& spend_tx);
     virtual void on_skip_transaction(uint64_t height, const cryptonote::transaction& tx);
     //----------------------------------------------------------
 

src/wallet/api/transaction_history.cpp

@@ -165,9 +165,8 @@ void TransactionHistoryImpl::refresh()
     for (std::list<std::pair<crypto::hash, tools::wallet2::unconfirmed_transfer_details>>::const_iterator i = upayments.begin(); i != upayments.end(); ++i) {
         const tools::wallet2::unconfirmed_transfer_details &pd = i->second;
         const crypto::hash &hash = i->first;
-        uint64_t amount = 0;
+        uint64_t amount = pd.m_amount_in;
-        cryptonote::get_inputs_money_amount(pd.m_tx, amount);
+        uint64_t fee = amount - pd.m_amount_out;
-        uint64_t fee = amount - get_outs_money_amount(pd.m_tx);
         std::string payment_id = string_tools::pod_to_hex(i->second.m_payment_id);
         if (payment_id.substr(16).find_first_not_of('0') == std::string::npos)
             payment_id = payment_id.substr(0,16);

src/wallet/api/wallet.cpp

@@ -79,11 +79,10 @@ struct Wallet2CallbackImpl : public tools::i_wallet2_callback
         LOG_PRINT_L3(__FUNCTION__ << ": new block. height: " << height);
     }
 
-    virtual void on_money_received(uint64_t height, const cryptonote::transaction& tx, size_t out_index)
+    virtual void on_money_received(uint64_t height, const cryptonote::transaction& tx, uint64_t amount)
     {
 
         std::string tx_hash =  epee::string_tools::pod_to_hex(get_transaction_hash(tx));
-        uint64_t amount     = tx.vout[out_index].amount;
 
         LOG_PRINT_L3(__FUNCTION__ << ": money received. height:  " << height
                      << ", tx: " << tx_hash
@@ -94,12 +93,11 @@ struct Wallet2CallbackImpl : public tools::i_wallet2_callback
         }
     }
 
-    virtual void on_money_spent(uint64_t height, const cryptonote::transaction& in_tx, size_t out_index,
+    virtual void on_money_spent(uint64_t height, const cryptonote::transaction& in_tx, uint64_t amount,
                                 const cryptonote::transaction& spend_tx)
     {
         // TODO;
         std::string tx_hash = epee::string_tools::pod_to_hex(get_transaction_hash(spend_tx));
-        uint64_t amount = in_tx.vout[out_index].amount;
         LOG_PRINT_L3(__FUNCTION__ << ": money spent. height:  " << height
                      << ", tx: " << tx_hash
                      << ", amount: " << print_money(amount));
@@ -491,12 +489,6 @@ PendingTransaction *WalletImpl::createTransaction(const string &dst_addr, const
         }
 
         de.amount = amount;
-        if (de.amount <= 0) {
-            m_status = Status_Error;
-            m_errorString = "Invalid amount";
-            break;
-        }
-
         dsts.push_back(de);
         //std::vector<tools::wallet2::pending_tx> ptx_vector;
 

src/wallet/wallet2.h

@@ -31,6 +31,7 @@
 #pragma once
 
 #include <memory>
+#include <boost/archive/binary_iarchive.hpp>
 #include <boost/serialization/list.hpp>
 #include <boost/serialization/vector.hpp>
 #include <atomic>
@@ -46,6 +47,8 @@
 #include "common/unordered_containers_boost_serialization.h"
 #include "crypto/chacha8.h"
 #include "crypto/hash.h"
+#include "ringct/rctTypes.h"
+#include "ringct/rctOps.h"
 
 #include "wallet_errors.h"
 
@@ -58,8 +61,8 @@ namespace tools
   {
   public:
     virtual void on_new_block(uint64_t height, const cryptonote::block& block) {}
-    virtual void on_money_received(uint64_t height, const cryptonote::transaction& tx, size_t out_index) {}
+    virtual void on_money_received(uint64_t height, const cryptonote::transaction& tx, uint64_t amount) {}
-    virtual void on_money_spent(uint64_t height, const cryptonote::transaction& in_tx, size_t out_index, const cryptonote::transaction& spend_tx) {}
+    virtual void on_money_spent(uint64_t height, const cryptonote::transaction& in_tx, uint64_t amount, const cryptonote::transaction& spend_tx) {}
     virtual void on_skip_transaction(uint64_t height, const cryptonote::transaction& tx) {}
     virtual ~i_wallet2_callback() {}
   };
@@ -96,13 +99,19 @@ namespace tools
     struct transfer_details
     {
       uint64_t m_block_height;
-      cryptonote::transaction m_tx;
+      cryptonote::transaction_prefix m_tx;
+      crypto::hash m_txid;
       size_t m_internal_output_index;
       uint64_t m_global_output_index;
       bool m_spent;
+      uint64_t m_spent_height;
       crypto::key_image m_key_image; //TODO: key_image stored twice :(
+      rct::key m_mask;
+      uint64_t m_amount;
+      bool m_rct;
 
-      uint64_t amount() const { return m_tx.vout[m_internal_output_index].amount; }
+      bool is_rct() const { return m_rct; }
+      uint64_t amount() const { return m_amount; }
     };
 
     struct payment_details
@@ -116,7 +125,9 @@ namespace tools
 
     struct unconfirmed_transfer_details
     {
-      cryptonote::transaction m_tx;
+      cryptonote::transaction_prefix m_tx;
+      uint64_t m_amount_in;
+      uint64_t m_amount_out;
       uint64_t m_change;
       time_t m_sent_time;
       std::vector<cryptonote::tx_destination_entry> m_dests;
@@ -137,7 +148,7 @@ namespace tools
 
       confirmed_transfer_details(): m_amount_in(0), m_amount_out(0), m_change((uint64_t)-1), m_block_height(0), m_payment_id(cryptonote::null_hash) {}
       confirmed_transfer_details(const unconfirmed_transfer_details &utd, uint64_t height):
-        m_amount_out(get_outs_money_amount(utd.m_tx)), m_change(utd.m_change), m_block_height(height), m_dests(utd.m_dests), m_payment_id(utd.m_payment_id), m_timestamp(utd.m_timestamp) { get_inputs_money_amount(utd.m_tx, m_amount_in); }
+        m_amount_in(utd.m_amount_in), m_amount_out(utd.m_amount_out), m_change(utd.m_change), m_block_height(height), m_dests(utd.m_dests), m_payment_id(utd.m_payment_id), m_timestamp(utd.m_timestamp) {}
     };
 
     typedef std::vector<transfer_details> transfer_container;
@@ -289,6 +300,8 @@ namespace tools
     template<typename T>
     void transfer_selected(const std::vector<cryptonote::tx_destination_entry>& dsts, const std::list<transfer_container::iterator> selected_transfers, size_t fake_outputs_count,
       uint64_t unlock_time, uint64_t fee, const std::vector<uint8_t>& extra, T destination_split_strategy, const tx_dust_policy& dust_policy, cryptonote::transaction& tx, pending_tx &ptx);
+    void transfer_selected_rct(std::vector<cryptonote::tx_destination_entry> dsts, const std::list<transfer_container::iterator> selected_transfers, size_t fake_outputs_count,
+      uint64_t unlock_time, uint64_t fee, const std::vector<uint8_t>& extra, cryptonote::transaction& tx, pending_tx &ptx);
 
     void commit_tx(pending_tx& ptx_vector);
     void commit_tx(std::vector<pending_tx>& ptx_vector);
@@ -308,6 +321,7 @@ namespace tools
     uint64_t get_blockchain_current_height() const { return m_local_bc_height; }
     void rescan_spent();
     void rescan_blockchain(bool refresh = true);
+    bool is_transfer_unlocked(const transfer_details& td) const;
     template <class t_archive>
     inline void serialize(t_archive &a, const unsigned int ver)
     {
@@ -339,6 +353,8 @@ namespace tools
       if(ver < 13)
         return;
       a & m_unconfirmed_payments;
+      if(ver < 14)
+        return;
     }
 
     /*!
@@ -376,8 +392,10 @@ namespace tools
 
     bool get_tx_key(const crypto::hash &txid, crypto::secret_key &tx_key) const;
 
+    uint64_t get_num_rct_outputs();
 
-    bool use_fork_rules(uint8_t version);
+    void get_hard_fork_info(uint8_t version, uint64_t &earliest_height);
+    bool use_fork_rules(uint8_t version, uint64_t early_blocks = 0);
 
     std::string get_wallet_file() const;
     std::string get_keys_file() const;
@@ -388,6 +406,9 @@ namespace tools
     std::vector<size_t> select_available_unmixable_outputs(bool trusted_daemon);
     std::vector<size_t> select_available_mixable_outputs(bool trusted_daemon);
 
+    size_t pop_best_value_from(const transfer_container &transfers, std::vector<size_t> &unused_dust_indices, const std::list<transfer_container::iterator>& selected_transfers) const;
+    size_t pop_best_value(std::vector<size_t> &unused_dust_indices, const std::list<transfer_container::iterator>& selected_transfers) const;
+
     void set_tx_note(const crypto::hash &txid, const std::string &note);
     std::string get_tx_note(const crypto::hash &txid) const;
 
@@ -413,32 +434,37 @@ namespace tools
      * \param password       Password of wallet file
      */
     bool load_keys(const std::string& keys_file_name, const std::string& password);
-    void process_new_transaction(const cryptonote::transaction& tx, uint64_t height, uint64_t ts, bool miner_tx, bool pool);
+    void process_new_transaction(const cryptonote::transaction& tx, const std::vector<uint64_t> &o_indices, uint64_t height, uint64_t ts, bool miner_tx, bool pool);
-    void process_new_blockchain_entry(const cryptonote::block& b, const cryptonote::block_complete_entry& bche, const crypto::hash& bl_id, uint64_t height);
+    void process_new_blockchain_entry(const cryptonote::block& b, const cryptonote::block_complete_entry& bche, const crypto::hash& bl_id, uint64_t height, const cryptonote::COMMAND_RPC_GET_BLOCKS_FAST::block_output_indices &o_indices);
     void detach_blockchain(uint64_t height);
     void get_short_chain_history(std::list<crypto::hash>& ids) const;
     bool is_tx_spendtime_unlocked(uint64_t unlock_time, uint64_t block_height) const;
-    bool is_transfer_unlocked(const transfer_details& td) const;
     bool clear();
-    void pull_blocks(uint64_t start_height, uint64_t& blocks_start_height, const std::list<crypto::hash> &short_chain_history, std::list<cryptonote::block_complete_entry> &blocks);
+    void pull_blocks(uint64_t start_height, uint64_t& blocks_start_height, const std::list<crypto::hash> &short_chain_history, std::list<cryptonote::block_complete_entry> &blocks, std::vector<cryptonote::COMMAND_RPC_GET_BLOCKS_FAST::block_output_indices> &o_indices);
     void pull_hashes(uint64_t start_height, uint64_t& blocks_start_height, const std::list<crypto::hash> &short_chain_history, std::list<crypto::hash> &hashes);
     void fast_refresh(uint64_t stop_height, uint64_t &blocks_start_height, std::list<crypto::hash> &short_chain_history);
-    void pull_next_blocks(uint64_t start_height, uint64_t &blocks_start_height, std::list<crypto::hash> &short_chain_history, const std::list<cryptonote::block_complete_entry> &prev_blocks, std::list<cryptonote::block_complete_entry> &blocks, bool &error);
+    void pull_next_blocks(uint64_t start_height, uint64_t &blocks_start_height, std::list<crypto::hash> &short_chain_history, const std::list<cryptonote::block_complete_entry> &prev_blocks, std::list<cryptonote::block_complete_entry> &blocks, std::vector<cryptonote::COMMAND_RPC_GET_BLOCKS_FAST::block_output_indices> &o_indices, bool &error);
-    void process_blocks(uint64_t start_height, const std::list<cryptonote::block_complete_entry> &blocks, uint64_t& blocks_added);
+    void process_blocks(uint64_t start_height, const std::list<cryptonote::block_complete_entry> &blocks, const std::vector<cryptonote::COMMAND_RPC_GET_BLOCKS_FAST::block_output_indices> &o_indices, uint64_t& blocks_added);
     uint64_t select_transfers(uint64_t needed_money, std::vector<size_t> unused_transfers_indices, std::list<transfer_container::iterator>& selected_transfers, bool trusted_daemon);
     bool prepare_file_names(const std::string& file_path);
     void process_unconfirmed(const cryptonote::transaction& tx, uint64_t height);
     void process_outgoing(const cryptonote::transaction& tx, uint64_t height, uint64_t ts, uint64_t spent, uint64_t received);
-    void add_unconfirmed_tx(const cryptonote::transaction& tx, const std::vector<cryptonote::tx_destination_entry> &dests, const crypto::hash &payment_id, uint64_t change_amount);
+    void add_unconfirmed_tx(const cryptonote::transaction& tx, uint64_t amount_in, const std::vector<cryptonote::tx_destination_entry> &dests, const crypto::hash &payment_id, uint64_t change_amount);
     void generate_genesis(cryptonote::block& b);
     void check_genesis(const crypto::hash& genesis_hash) const; //throws
     bool generate_chacha8_key_from_secret_keys(crypto::chacha8_key &key) const;
     crypto::hash get_payment_id(const pending_tx &ptx) const;
-    void check_acc_out(const cryptonote::account_keys &acc, const cryptonote::tx_out &o, const crypto::public_key &tx_pub_key, size_t i, uint64_t &money_transfered, bool &error) const;
+    void check_acc_out(const cryptonote::account_keys &acc, const cryptonote::tx_out &o, const crypto::public_key &tx_pub_key, size_t i, bool &received, uint64_t &money_transfered, bool &error) const;
     void parse_block_round(const cryptonote::blobdata &blob, cryptonote::block &bl, crypto::hash &bl_id, bool &error) const;
     uint64_t get_upper_tranaction_size_limit();
     std::vector<uint64_t> get_unspent_amounts_vector();
     uint64_t sanitize_fee_multiplier(uint64_t fee_multiplier) const;
+    float get_output_relatedness(const transfer_details &td0, const transfer_details &td1) const;
+    std::vector<size_t> pick_prefered_rct_inputs(uint64_t needed_money) const;
+    void set_spent(transfer_details &td, uint64_t height);
+    void set_unspent(transfer_details &td);
+    template<typename entry>
+    void get_outs(std::vector<std::vector<entry>> &outs, const std::list<transfer_container::iterator> &selected_transfers, size_t fake_outputs_count);
 
     cryptonote::account_base m_account;
     std::string m_daemon_address;
@@ -478,24 +504,83 @@ namespace tools
     uint64_t m_refresh_from_block_height;
   };
 }
-BOOST_CLASS_VERSION(tools::wallet2, 13)
+BOOST_CLASS_VERSION(tools::wallet2, 14)
+BOOST_CLASS_VERSION(tools::wallet2::transfer_details, 4)
 BOOST_CLASS_VERSION(tools::wallet2::payment_details, 1)
-BOOST_CLASS_VERSION(tools::wallet2::unconfirmed_transfer_details, 3)
+BOOST_CLASS_VERSION(tools::wallet2::unconfirmed_transfer_details, 5)
 BOOST_CLASS_VERSION(tools::wallet2::confirmed_transfer_details, 2)
 
 namespace boost
 {
   namespace serialization
   {
+    template <class Archive>
+    inline void initialize_transfer_details(Archive &a, tools::wallet2::transfer_details &x, const boost::serialization::version_type ver)
+    {
+    }
+    template<>
+    inline void initialize_transfer_details(boost::archive::binary_iarchive &a, tools::wallet2::transfer_details &x, const boost::serialization::version_type ver)
+    {
+        if (ver < 1)
+        {
+          x.m_mask = rct::identity();
+          x.m_amount = x.m_tx.vout[x.m_internal_output_index].amount;
+        }
+        if (ver < 2)
+        {
+          x.m_spent_height = 0;
+        }
+        if (ver < 4)
+        {
+          x.m_rct = x.m_tx.vout[x.m_internal_output_index].amount == 0;
+        }
+    }
+
     template <class Archive>
     inline void serialize(Archive &a, tools::wallet2::transfer_details &x, const boost::serialization::version_type ver)
     {
       a & x.m_block_height;
       a & x.m_global_output_index;
       a & x.m_internal_output_index;
+      if (ver < 3)
+      {
+        cryptonote::transaction tx;
+        a & tx;
+        x.m_tx = (const cryptonote::transaction_prefix&)tx;
+        x.m_txid = cryptonote::get_transaction_hash(tx);
+      }
+      else
+      {
         a & x.m_tx;
+      }
       a & x.m_spent;
       a & x.m_key_image;
+      if (ver < 1)
+      {
+        // ensure mask and amount are set
+        initialize_transfer_details(a, x, ver);
+        return;
+      }
+      a & x.m_mask;
+      a & x.m_amount;
+      if (ver < 2)
+      {
+        initialize_transfer_details(a, x, ver);
+        return;
+      }
+      a & x.m_spent_height;
+      if (ver < 3)
+      {
+        initialize_transfer_details(a, x, ver);
+        return;
+      }
+      a & x.m_txid;
+      if (ver < 4)
+      {
+        initialize_transfer_details(a, x, ver);
+        return;
+      }
+      a & x.m_rct;
     }
 
     template <class Archive>
@@ -503,7 +588,16 @@ namespace boost
     {
       a & x.m_change;
       a & x.m_sent_time;
+      if (ver < 5)
+      {
+        cryptonote::transaction tx;
+        a & tx;
+        x.m_tx = (const cryptonote::transaction_prefix&)tx;
+      }
+      else
+      {
         a & x.m_tx;
+      }
       if (ver < 1)
         return;
       a & x.m_dests;
@@ -514,6 +608,10 @@ namespace boost
       if (ver < 3)
         return;
       a & x.m_timestamp;
+      if (ver < 4)
+        return;
+      a & x.m_amount_in;
+      a & x.m_amount_out;
     }
 
     template <class Archive>
@@ -688,6 +786,7 @@ namespace tools
       cryptonote::tx_source_entry& src = sources.back();
       transfer_details& td = *it;
       src.amount = td.amount();
+      src.rct = false;
       //paste mixin transaction
       if(daemon_resp.outs.size())
       {
@@ -698,7 +797,8 @@ namespace tools
             continue;
           tx_output_entry oe;
           oe.first = daemon_oe.global_amount_index;
-          oe.second = daemon_oe.out_key;
+          oe.second.dest = rct::pk2rct(daemon_oe.out_key);
+          oe.second.mask = rct::identity();
           src.outputs.push_back(oe);
           if(src.outputs.size() >= fake_outputs_count)
             break;
@@ -713,7 +813,8 @@ namespace tools
       //size_t real_index = src.outputs.size() ? (rand() % src.outputs.size() ):0;
       tx_output_entry real_oe;
       real_oe.first = td.m_global_output_index;
-      real_oe.second = boost::get<txout_to_key>(td.m_tx.vout[td.m_internal_output_index].target).key;
+      real_oe.second.dest = rct::pk2rct(boost::get<txout_to_key>(td.m_tx.vout[td.m_internal_output_index].target).key);
+      real_oe.second.mask = rct::identity();
       auto interted_it = src.outputs.insert(it_to_insert, real_oe);
       src.real_out_tx_key = get_tx_pub_key_from_extra(td.m_tx);
       src.real_output = interted_it - src.outputs.begin();

src/wallet/wallet_rpc_server.cpp

@@ -235,11 +235,11 @@ namespace tools
     try
     {
       uint64_t mixin = req.mixin;
-      if (mixin < 2 && m_wallet.use_fork_rules(2)) {
+      if (mixin < 2 && m_wallet.use_fork_rules(2, 10)) {
         LOG_PRINT_L1("Requested mixin " << req.mixin << " too low for hard fork 2, using 2");
         mixin = 2;
       }
-      std::vector<wallet2::pending_tx> ptx_vector = m_wallet.create_transactions(dsts, mixin, req.unlock_time, req.fee_multiplier, extra, req.trusted_daemon);
+      std::vector<wallet2::pending_tx> ptx_vector = m_wallet.create_transactions_2(dsts, mixin, req.unlock_time, req.fee_multiplier, extra, req.trusted_daemon);
 
       // reject proposed transactions if there are more than one.  see on_transfer_split below.
       if (ptx_vector.size() != 1)
@@ -254,7 +254,9 @@ namespace tools
       // populate response with tx hash
       res.tx_hash = epee::string_tools::pod_to_hex(cryptonote::get_transaction_hash(ptx_vector.back().tx));
       if (req.get_tx_key)
+      {
         res.tx_key = epee::string_tools::pod_to_hex(ptx_vector.back().tx_key);
+      }
       return true;
     }
     catch (const tools::error::daemon_busy& e)
@@ -300,15 +302,12 @@ namespace tools
     try
     {
       uint64_t mixin = req.mixin;
-      if (mixin < 2 && m_wallet.use_fork_rules(2)) {
+      if (mixin < 2 && m_wallet.use_fork_rules(2, 10)) {
         LOG_PRINT_L1("Requested mixin " << req.mixin << " too low for hard fork 2, using 2");
         mixin = 2;
       }
       std::vector<wallet2::pending_tx> ptx_vector;
-      if (req.new_algorithm)
       ptx_vector = m_wallet.create_transactions_2(dsts, mixin, req.unlock_time, req.fee_multiplier, extra, req.trusted_daemon);
-      else
-        ptx_vector = m_wallet.create_transactions(dsts, mixin, req.unlock_time, req.fee_multiplier, extra, req.trusted_daemon);
 
       m_wallet.commit_tx(ptx_vector);
 
@@ -317,8 +316,10 @@ namespace tools
       {
         res.tx_hash_list.push_back(epee::string_tools::pod_to_hex(cryptonote::get_transaction_hash(ptx.tx)));
         if (req.get_tx_keys)
+        {
           res.tx_key_list.push_back(epee::string_tools::pod_to_hex(ptx.tx_key));
         }
+      }
 
       return true;
     }
@@ -363,8 +364,10 @@ namespace tools
       {
         res.tx_hash_list.push_back(epee::string_tools::pod_to_hex(cryptonote::get_transaction_hash(ptx.tx)));
         if (req.get_tx_keys)
+        {
           res.tx_key_list.push_back(epee::string_tools::pod_to_hex(ptx.tx_key));
         }
+      }
 
       return true;
     }
@@ -422,8 +425,10 @@ namespace tools
       {
         res.tx_hash_list.push_back(epee::string_tools::pod_to_hex(cryptonote::get_transaction_hash(ptx.tx)));
         if (req.get_tx_keys)
+        {
           res.tx_key_list.push_back(epee::string_tools::pod_to_hex(ptx.tx_key));
         }
+      }
 
       return true;
     }
@@ -693,7 +698,7 @@ namespace tools
         rpc_transfers.amount       = td.amount();
         rpc_transfers.spent        = td.m_spent;
         rpc_transfers.global_index = td.m_global_output_index;
-        rpc_transfers.tx_hash      = epee::string_tools::pod_to_hex(cryptonote::get_transaction_hash(td.m_tx));
+        rpc_transfers.tx_hash      = epee::string_tools::pod_to_hex(td.m_txid);
         rpc_transfers.tx_size      = txBlob.size();
         res.transfers.push_back(rpc_transfers);
       }
@@ -961,10 +966,8 @@ namespace tools
           entry.payment_id = entry.payment_id.substr(0,16);
         entry.height = 0;
         entry.timestamp = pd.m_timestamp;
-        uint64_t amount = 0;
+        entry.fee = pd.m_amount_in - pd.m_amount_out;
-        cryptonote::get_inputs_money_amount(pd.m_tx, amount);
+        entry.amount = pd.m_amount_in - pd.m_change - entry.fee;
-        entry.fee = amount - get_outs_money_amount(pd.m_tx);
-        entry.amount = amount - pd.m_change - entry.fee;
         entry.note = m_wallet.get_tx_note(i->first);
       }
     }

src/wallet/wallet_rpc_server_commands_defs.h

@@ -132,10 +132,12 @@ namespace wallet_rpc
     {
       std::string tx_hash;
       std::string tx_key;
+      std::list<std::string> amount_keys;
 
       BEGIN_KV_SERIALIZE_MAP()
         KV_SERIALIZE(tx_hash)
         KV_SERIALIZE(tx_key)
+        KV_SERIALIZE(amount_keys)
       END_KV_SERIALIZE_MAP()
     };
   };
@@ -149,7 +151,6 @@ namespace wallet_rpc
       uint64_t mixin;
       uint64_t unlock_time;
       std::string payment_id;
-      bool new_algorithm;
       bool get_tx_keys;
       bool trusted_daemon;
 
@@ -159,12 +160,20 @@ namespace wallet_rpc
         KV_SERIALIZE(mixin)
         KV_SERIALIZE(unlock_time)
         KV_SERIALIZE(payment_id)
-        KV_SERIALIZE(new_algorithm)
         KV_SERIALIZE(get_tx_keys)
         KV_SERIALIZE(trusted_daemon)
       END_KV_SERIALIZE_MAP()
     };
 
+    struct key_list
+    {
+      std::list<std::string> keys;
+
+      BEGIN_KV_SERIALIZE_MAP()
+        KV_SERIALIZE(keys)
+      END_KV_SERIALIZE_MAP()
+    };
+
     struct response
     {
       std::list<std::string> tx_hash_list;
@@ -190,6 +199,15 @@ namespace wallet_rpc
       END_KV_SERIALIZE_MAP()
     };
 
+    struct key_list
+    {
+      std::list<std::string> keys;
+
+      BEGIN_KV_SERIALIZE_MAP()
+        KV_SERIALIZE(keys)
+      END_KV_SERIALIZE_MAP()
+    };
+
     struct response
     {
       std::list<std::string> tx_hash_list;
@@ -225,6 +243,15 @@ namespace wallet_rpc
       END_KV_SERIALIZE_MAP()
     };
 
+    struct key_list
+    {
+      std::list<std::string> keys;
+
+      BEGIN_KV_SERIALIZE_MAP()
+        KV_SERIALIZE(keys)
+      END_KV_SERIALIZE_MAP()
+    };
+
     struct response
     {
       std::list<std::string> tx_hash_list;

tests/core_proxy/core_proxy.h

@@ -34,7 +34,6 @@
 
 #include "cryptonote_core/cryptonote_basic_impl.h"
 #include "cryptonote_core/verification_context.h"
-#include "cryptonote_core/blockchain_storage.h"
 #include <unordered_map>
 
 namespace tests
@@ -82,7 +81,7 @@ namespace tests
     bool on_idle(){return true;}
     bool find_blockchain_supplement(const std::list<crypto::hash>& qblock_ids, cryptonote::NOTIFY_RESPONSE_CHAIN_ENTRY::request& resp){return true;}
     bool handle_get_objects(cryptonote::NOTIFY_REQUEST_GET_OBJECTS::request& arg, cryptonote::NOTIFY_RESPONSE_GET_OBJECTS::request& rsp, cryptonote::cryptonote_connection_context& context){return true;}
-    cryptonote::blockchain_storage &get_blockchain_storage() { throw std::runtime_error("Called invalid member function: please never call get_blockchain_storage on the TESTING class proxy_core."); }
+    cryptonote::Blockchain &get_blockchain_storage() { throw std::runtime_error("Called invalid member function: please never call get_blockchain_storage on the TESTING class proxy_core."); }
     bool get_test_drop_download() {return true;}
     bool get_test_drop_download_height() {return true;}
     bool prepare_handle_incoming_blocks(const std::list<cryptonote::block_complete_entry>  &blocks) { return true; }

tests/core_tests/CMakeLists.txt

@@ -39,7 +39,8 @@ set(core_tests_sources
   ring_signature_1.cpp
   transaction_tests.cpp
   tx_validation.cpp
-  v2_tests.cpp)
+  v2_tests.cpp
+  rct.cpp)
 
 set(core_tests_headers
   block_reward.h
@@ -54,7 +55,8 @@ set(core_tests_headers
   ring_signature_1.h
   transaction_tests.h
   tx_validation.h
-  v2_tests.h)
+  v2_tests.h
+  rct.h)
 
 add_executable(coretests
   ${core_tests_sources}

tests/core_tests/block_validation.cpp

@@ -335,8 +335,9 @@ bool gen_block_miner_tx_has_2_in::generate(std::vector<test_event_entry>& events
 
   tx_source_entry se;
   se.amount = blk_0.miner_tx.vout[0].amount;
-  se.outputs.push_back(std::make_pair(0, boost::get<txout_to_key>(blk_0.miner_tx.vout[0].target).key));
+  se.push_output(0, boost::get<txout_to_key>(blk_0.miner_tx.vout[0].target).key, se.amount);
   se.real_output = 0;
+  se.rct = false;
   se.real_out_tx_key = get_tx_pub_key_from_extra(blk_0.miner_tx);
   se.real_output_in_tx_index = 0;
   std::vector<tx_source_entry> sources;
@@ -377,8 +378,9 @@ bool gen_block_miner_tx_with_txin_to_key::generate(std::vector<test_event_entry>
 
   tx_source_entry se;
   se.amount = blk_1.miner_tx.vout[0].amount;
-  se.outputs.push_back(std::make_pair(0, boost::get<txout_to_key>(blk_1.miner_tx.vout[0].target).key));
+  se.push_output(0, boost::get<txout_to_key>(blk_1.miner_tx.vout[0].target).key, se.amount);
   se.real_output = 0;
+  se.rct = false;
   se.real_out_tx_key = get_tx_pub_key_from_extra(blk_1.miner_tx);
   se.real_output_in_tx_index = 0;
   std::vector<tx_source_entry> sources;

tests/core_tests/chaingen.cpp

@@ -94,11 +94,11 @@ uint64_t test_generator::get_already_generated_coins(const cryptonote::block& bl
   return get_already_generated_coins(blk_hash);
 }
 
-void test_generator::add_block(const cryptonote::block& blk, size_t tsx_size, std::vector<size_t>& block_sizes, uint64_t already_generated_coins)
+void test_generator::add_block(const cryptonote::block& blk, size_t tsx_size, std::vector<size_t>& block_sizes, uint64_t already_generated_coins, uint8_t hf_version)
 {
   const size_t block_size = tsx_size + get_object_blobsize(blk.miner_tx);
   uint64_t block_reward;
-  get_block_reward(misc_utils::median(block_sizes), block_size, already_generated_coins, block_reward, 1);
+  get_block_reward(misc_utils::median(block_sizes), block_size, already_generated_coins, block_reward, hf_version);
   m_blocks_info[get_block_hash(blk)] = block_info(blk.prev_id, already_generated_coins + block_reward, block_size);
 }
 
@@ -215,7 +215,7 @@ bool test_generator::construct_block_manually(block& blk, const block& prev_bloc
                                               const crypto::hash& prev_id/* = crypto::hash()*/, const difficulty_type& diffic/* = 1*/,
                                               const transaction& miner_tx/* = transaction()*/,
                                               const std::vector<crypto::hash>& tx_hashes/* = std::vector<crypto::hash>()*/,
-                                              size_t txs_sizes/* = 0*/, size_t max_outs/* = 0*/)
+                                              size_t txs_sizes/* = 0*/, size_t max_outs/* = 0*/, uint8_t hf_version/* = 1*/)
 {
   blk.major_version = actual_params & bf_major_ver ? major_ver : CURRENT_BLOCK_MAJOR_VERSION;
   blk.minor_version = actual_params & bf_minor_ver ? minor_ver : CURRENT_BLOCK_MINOR_VERSION;
@@ -223,6 +223,7 @@ bool test_generator::construct_block_manually(block& blk, const block& prev_bloc
   blk.prev_id       = actual_params & bf_prev_id   ? prev_id   : get_block_hash(prev_block);
   blk.tx_hashes     = actual_params & bf_tx_hashes ? tx_hashes : std::vector<crypto::hash>();
   max_outs          = actual_params & bf_max_outs ? max_outs : 9999;
+  hf_version        = actual_params & bf_hf_version ? hf_version : 1;
 
   size_t height = get_block_height(prev_block) + 1;
   uint64_t already_generated_coins = get_already_generated_coins(prev_block);
@@ -236,7 +237,7 @@ bool test_generator::construct_block_manually(block& blk, const block& prev_bloc
   {
     size_t current_block_size = txs_sizes + get_object_blobsize(blk.miner_tx);
     // TODO: This will work, until size of constructed block is less then CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE
-    if (!construct_miner_tx(height, misc_utils::median(block_sizes), already_generated_coins, current_block_size, 0, miner_acc.get_keys().m_account_address, blk.miner_tx, blobdata(), max_outs))
+    if (!construct_miner_tx(height, misc_utils::median(block_sizes), already_generated_coins, current_block_size, 0, miner_acc.get_keys().m_account_address, blk.miner_tx, blobdata(), max_outs, hf_version))
       return false;
   }
 
@@ -245,7 +246,7 @@ bool test_generator::construct_block_manually(block& blk, const block& prev_bloc
   difficulty_type a_diffic = actual_params & bf_diffic ? diffic : get_test_difficulty();
   fill_nonce(blk, a_diffic, height);
 
-  add_block(blk, txs_sizes, block_sizes, already_generated_coins);
+  add_block(blk, txs_sizes, block_sizes, already_generated_coins, hf_version);
 
   return true;
 }
@@ -412,7 +413,7 @@ bool fill_output_entries(std::vector<output_index>& out_indices, size_t sender_o
     if (append)
     {
       const txout_to_key& otk = boost::get<txout_to_key>(oi.out);
-      output_entries.push_back(tx_source_entry::output_entry(oi.idx, otk.key));
+      output_entries.push_back(tx_source_entry::output_entry(oi.idx, rct::ctkey({rct::pk2rct(otk.key), rct::identity()})));
     }
   }
 
@@ -457,6 +458,7 @@ bool fill_tx_sources(std::vector<tx_source_entry>& sources, const std::vector<te
               continue;
 
             ts.real_output = realOutput;
+            ts.rct = false;
 
             sources.push_back(ts);
 
@@ -544,7 +546,7 @@ bool construct_miner_tx_manually(size_t height, uint64_t already_generated_coins
   out.target = txout_to_key(out_eph_public_key);
   tx.vout.push_back(out);
 
-  tx.version = CURRENT_TRANSACTION_VERSION;
+  tx.version = 1;
   tx.unlock_time = height + CRYPTONOTE_MINED_MONEY_UNLOCK_WINDOW;
 
   return true;

tests/core_tests/chaingen.h

@@ -221,7 +221,8 @@ public:
     bf_miner_tx  = 1 << 4,
     bf_tx_hashes = 1 << 5,
     bf_diffic    = 1 << 6,
-    bf_max_outs  = 1 << 7
+    bf_max_outs  = 1 << 7,
+    bf_hf_version= 1 << 8
   };
 
   void get_block_chain(std::vector<block_info>& blockchain, const crypto::hash& head, size_t n) const;
@@ -229,7 +230,8 @@ public:
   uint64_t get_already_generated_coins(const crypto::hash& blk_id) const;
   uint64_t get_already_generated_coins(const cryptonote::block& blk) const;
 
-  void add_block(const cryptonote::block& blk, size_t tsx_size, std::vector<size_t>& block_sizes, uint64_t already_generated_coins);
+  void add_block(const cryptonote::block& blk, size_t tsx_size, std::vector<size_t>& block_sizes, uint64_t already_generated_coins,
+    uint8_t hf_version = 1);
   bool construct_block(cryptonote::block& blk, uint64_t height, const crypto::hash& prev_id,
     const cryptonote::account_base& miner_acc, uint64_t timestamp, uint64_t already_generated_coins,
     std::vector<size_t>& block_sizes, const std::list<cryptonote::transaction>& tx_list);
@@ -241,7 +243,8 @@ public:
     const cryptonote::account_base& miner_acc, int actual_params = bf_none, uint8_t major_ver = 0,
     uint8_t minor_ver = 0, uint64_t timestamp = 0, const crypto::hash& prev_id = crypto::hash(),
     const cryptonote::difficulty_type& diffic = 1, const cryptonote::transaction& miner_tx = cryptonote::transaction(),
-    const std::vector<crypto::hash>& tx_hashes = std::vector<crypto::hash>(), size_t txs_sizes = 0, size_t max_outs = 999);
+    const std::vector<crypto::hash>& tx_hashes = std::vector<crypto::hash>(), size_t txs_sizes = 0, size_t max_outs = 999,
+    uint8_t hf_version = 1);
   bool construct_block_manually_tx(cryptonote::block& blk, const cryptonote::block& prev_block,
     const cryptonote::account_base& miner_acc, const std::vector<crypto::hash>& tx_hashes, size_t txs_size);
 
@@ -472,11 +475,11 @@ inline bool replay_events_through_core(cryptonote::core& cr, const std::vector<t
 //--------------------------------------------------------------------------
 template<typename t_test_class>
 struct get_test_options {
-  const std::pair<uint8_t, uint64_t> hard_forks[1];
+  const std::pair<uint8_t, uint64_t> hard_forks[2];
   const cryptonote::test_options test_options = {
     hard_forks
   };
-  get_test_options():hard_forks{std::make_pair((uint8_t)1, (uint64_t)0)}{}
+  get_test_options():hard_forks{std::make_pair((uint8_t)1, (uint64_t)0), std::make_pair((uint8_t)0, (uint64_t)0)}{}
 };
 
 //--------------------------------------------------------------------------

tests/core_tests/chaingen_main.cpp

@@ -173,6 +173,35 @@ int main(int argc, char* argv[])
 //    GENERATE_AND_PLAY(gen_v2_tx_unmixable_two);
     GENERATE_AND_PLAY(gen_v2_tx_dust);
 
+    GENERATE_AND_PLAY(gen_rct_tx_valid_from_pre_rct);
+    GENERATE_AND_PLAY(gen_rct_tx_valid_from_rct);
+    GENERATE_AND_PLAY(gen_rct_tx_valid_from_mixed);
+    GENERATE_AND_PLAY(gen_rct_tx_pre_rct_bad_real_dest);
+    GENERATE_AND_PLAY(gen_rct_tx_pre_rct_bad_real_mask);
+    GENERATE_AND_PLAY(gen_rct_tx_pre_rct_bad_fake_dest);
+    GENERATE_AND_PLAY(gen_rct_tx_pre_rct_bad_fake_mask);
+    GENERATE_AND_PLAY(gen_rct_tx_rct_bad_real_dest);
+    GENERATE_AND_PLAY(gen_rct_tx_rct_bad_real_mask);
+    GENERATE_AND_PLAY(gen_rct_tx_rct_bad_fake_dest);
+    GENERATE_AND_PLAY(gen_rct_tx_rct_bad_fake_mask);
+    GENERATE_AND_PLAY(gen_rct_tx_rct_spend_with_zero_commit);
+    GENERATE_AND_PLAY(gen_rct_tx_pre_rct_zero_vin_amount);
+    GENERATE_AND_PLAY(gen_rct_tx_rct_non_zero_vin_amount);
+    GENERATE_AND_PLAY(gen_rct_tx_non_zero_vout_amount);
+    GENERATE_AND_PLAY(gen_rct_tx_pre_rct_duplicate_key_image);
+    GENERATE_AND_PLAY(gen_rct_tx_rct_duplicate_key_image);
+    GENERATE_AND_PLAY(gen_rct_tx_pre_rct_wrong_key_image);
+    GENERATE_AND_PLAY(gen_rct_tx_rct_wrong_key_image);
+    GENERATE_AND_PLAY(gen_rct_tx_pre_rct_wrong_fee);
+    GENERATE_AND_PLAY(gen_rct_tx_rct_wrong_fee);
+    GENERATE_AND_PLAY(gen_rct_tx_pre_rct_remove_vin);
+    GENERATE_AND_PLAY(gen_rct_tx_rct_remove_vin);
+    GENERATE_AND_PLAY(gen_rct_tx_pre_rct_add_vout);
+    GENERATE_AND_PLAY(gen_rct_tx_rct_add_vout);
+    GENERATE_AND_PLAY(gen_rct_tx_pre_rct_increase_vin_and_fee);
+    GENERATE_AND_PLAY(gen_rct_tx_pre_rct_altered_extra);
+    GENERATE_AND_PLAY(gen_rct_tx_rct_altered_extra);
+
     std::cout << (failed_tests.empty() ? concolor::green : concolor::magenta);
     std::cout << "\nREPORT:\n";
     std::cout << "  Test run: " << tests_count << '\n';

tests/core_tests/chaingen_tests_list.h

@@ -40,6 +40,7 @@
 #include "ring_signature_1.h"
 #include "tx_validation.h"
 #include "v2_tests.h"
+#include "rct.h"
 /************************************************************************/
 /*                                                                      */
 /************************************************************************/

tests/core_tests/double_spend.inl

@@ -128,8 +128,9 @@ bool gen_double_spend_in_tx<txs_keeped_by_block>::generate(std::vector<test_even
   std::vector<cryptonote::tx_source_entry> sources;
   cryptonote::tx_source_entry se;
   se.amount = tx_0.vout[0].amount;
-  se.outputs.push_back(std::make_pair(0, boost::get<cryptonote::txout_to_key>(tx_0.vout[0].target).key));
+  se.push_output(0, boost::get<cryptonote::txout_to_key>(tx_0.vout[0].target).key, se.amount);
   se.real_output = 0;
+  se.rct = false;
   se.real_out_tx_key = get_tx_pub_key_from_extra(tx_0);
   se.real_output_in_tx_index = 0;
   sources.push_back(se);

tests/core_tests/integer_overflow.cpp

@@ -61,8 +61,9 @@ namespace
   {
     cryptonote::tx_source_entry se;
     se.amount = tx.vout[out_idx].amount;
-    se.outputs.push_back(std::make_pair(0, boost::get<cryptonote::txout_to_key>(tx.vout[out_idx].target).key));
+    se.push_output(0, boost::get<cryptonote::txout_to_key>(tx.vout[out_idx].target).key, se.amount);
     se.real_output = 0;
+    se.rct = false;
     se.real_out_tx_key = get_tx_pub_key_from_extra(tx);
     se.real_output_in_tx_index = out_idx;
 

tests/core_tests/rct.cpp

@@ -0,0 +1,499 @@
+// Copyright (c) 2014-2016, The Monero Project
+// 
+// All rights reserved.
+// 
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+// 
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+// 
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+// 
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+// 
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+// 
+// Parts of this file are originally copyright (c) 2012-2013 The Cryptonote developers
+
+#include "ringct/rctSigs.h"
+#include "chaingen.h"
+#include "chaingen_tests_list.h"
+
+using namespace epee;
+using namespace crypto;
+using namespace cryptonote;
+
+//----------------------------------------------------------------------------------------------------------------------
+// Tests
+
+bool gen_rct_tx_validation_base::generate_with(std::vector<test_event_entry>& events,
+    const int *out_idx, int mixin, uint64_t amount_paid, bool valid,
+    const std::function<void(std::vector<tx_source_entry> &sources, std::vector<tx_destination_entry> &destinations)> &pre_tx,
+    const std::function<void(transaction &tx)> &post_tx) const
+{
+  uint64_t ts_start = 1338224400;
+
+  GENERATE_ACCOUNT(miner_account);
+  MAKE_GENESIS_BLOCK(events, blk_0, miner_account, ts_start);
+
+  // create 4 miner accounts, and have them mine the next 4 blocks
+  cryptonote::account_base miner_accounts[4];
+  const cryptonote::block *prev_block = &blk_0;
+  cryptonote::block blocks[4];
+  for (size_t n = 0; n < 4; ++n) {
+    miner_accounts[n].generate();
+    CHECK_AND_ASSERT_MES(generator.construct_block_manually(blocks[n], *prev_block, miner_accounts[n],
+        test_generator::bf_major_ver | test_generator::bf_minor_ver | test_generator::bf_timestamp | test_generator::bf_hf_version,
+        2, 2, prev_block->timestamp + DIFFICULTY_BLOCKS_ESTIMATE_TIMESPAN * 2, // v2 has blocks twice as long
+          crypto::hash(), 0, transaction(), std::vector<crypto::hash>(), 0, 0, 2),
+        false, "Failed to generate block");
+    events.push_back(blocks[n]);
+    prev_block = blocks + n;
+    LOG_PRINT_L0("Initial miner tx " << n << ": " << obj_to_json_str(blocks[n].miner_tx));
+  }
+
+  // rewind
+  cryptonote::block blk_r, blk_last;
+  {
+    blk_last = blocks[3];
+    for (size_t i = 0; i < CRYPTONOTE_MINED_MONEY_UNLOCK_WINDOW; ++i)
+    {
+      cryptonote::block blk;
+      CHECK_AND_ASSERT_MES(generator.construct_block_manually(blk, blk_last, miner_account,
+          test_generator::bf_major_ver | test_generator::bf_minor_ver | test_generator::bf_timestamp | test_generator::bf_hf_version,
+          2, 2, blk_last.timestamp + DIFFICULTY_BLOCKS_ESTIMATE_TIMESPAN * 2, // v2 has blocks twice as long
+          crypto::hash(), 0, transaction(), std::vector<crypto::hash>(), 0, 0, 2),
+          false, "Failed to generate block");
+      events.push_back(blk);
+      blk_last = blk;
+    }
+    blk_r = blk_last;
+  }
+
+  // create 4 txes from these miners in another block, to generate some rct outputs
+  transaction rct_txes[4];
+  rct::key rct_tx_masks[16];
+  cryptonote::block blk_txes[4];
+  for (size_t n = 0; n < 4; ++n)
+  {
+    std::vector<crypto::hash> starting_rct_tx_hashes;
+    std::vector<tx_source_entry> sources;
+
+    sources.resize(1);
+    tx_source_entry& src = sources.back();
+
+    const size_t index_in_tx = 5;
+    src.amount = 30000000000000;
+    for (int m = 0; m < 4; ++m) {
+      src.push_output(m, boost::get<txout_to_key>(blocks[m].miner_tx.vout[index_in_tx].target).key, src.amount);
+    }
+    src.real_out_tx_key = cryptonote::get_tx_pub_key_from_extra(blocks[n].miner_tx);
+    src.real_output = n;
+    src.real_output_in_tx_index = index_in_tx;
+    src.mask = rct::identity();
+    src.rct = false;
+
+    //fill outputs entry
+    tx_destination_entry td;
+    td.addr = miner_accounts[n].get_keys().m_account_address;
+    td.amount = 7390000000000;
+    std::vector<tx_destination_entry> destinations;
+    destinations.push_back(td);
+    destinations.push_back(td);
+    destinations.push_back(td);
+    destinations.push_back(td); // 30 -> 7.39 * 4
+
+    crypto::secret_key tx_key;
+    bool r = construct_tx_and_get_tx_key(miner_accounts[n].get_keys(), sources, destinations, std::vector<uint8_t>(), rct_txes[n], 0, tx_key, true);
+    CHECK_AND_ASSERT_MES(r, false, "failed to construct transaction");
+    events.push_back(rct_txes[n]);
+    starting_rct_tx_hashes.push_back(get_transaction_hash(rct_txes[n]));
+
+    for (size_t o = 0; o < 4; ++o)
+    {
+      crypto::key_derivation derivation;
+      bool r = crypto::generate_key_derivation(destinations[o].addr.m_view_public_key, tx_key, derivation);
+      CHECK_AND_ASSERT_MES(r, false, "Failed to generate key derivation");
+      crypto::secret_key amount_key;
+      crypto::derivation_to_scalar(derivation, o, amount_key);
+      if (rct_txes[n].rct_signatures.type == rct::RCTTypeSimple)
+        rct::decodeRctSimple(rct_txes[n].rct_signatures, rct::sk2rct(amount_key), o, rct_tx_masks[o+n*4]);
+      else
+        rct::decodeRct(rct_txes[n].rct_signatures, rct::sk2rct(amount_key), o, rct_tx_masks[o+n*4]);
+    }
+
+    CHECK_AND_ASSERT_MES(generator.construct_block_manually(blk_txes[n], blk_last, miner_account,
+        test_generator::bf_major_ver | test_generator::bf_minor_ver | test_generator::bf_timestamp | test_generator::bf_tx_hashes | test_generator::bf_hf_version | test_generator::bf_max_outs,
+        4, 4, blk_last.timestamp + DIFFICULTY_BLOCKS_ESTIMATE_TIMESPAN * 2, // v2 has blocks twice as long
+        crypto::hash(), 0, transaction(), starting_rct_tx_hashes, 0, 6, 4),
+        false, "Failed to generate block");
+    events.push_back(blk_txes[n]);
+    blk_last = blk_txes[n];
+  }
+
+  // rewind
+  {
+    for (size_t i = 0; i < CRYPTONOTE_MINED_MONEY_UNLOCK_WINDOW; ++i)
+    {
+      cryptonote::block blk;
+      CHECK_AND_ASSERT_MES(generator.construct_block_manually(blk, blk_last, miner_account,
+          test_generator::bf_major_ver | test_generator::bf_minor_ver | test_generator::bf_timestamp | test_generator::bf_hf_version | test_generator::bf_max_outs,
+          4, 4, blk_last.timestamp + DIFFICULTY_BLOCKS_ESTIMATE_TIMESPAN * 2, // v2 has blocks twice as long
+          crypto::hash(), 0, transaction(), std::vector<crypto::hash>(), 0, 6, 4),
+          false, "Failed to generate block");
+      events.push_back(blk);
+      blk_last = blk;
+    }
+    blk_r = blk_last;
+  }
+
+  // create a tx from the requested ouputs
+  std::vector<tx_source_entry> sources;
+  size_t global_rct_idx = 6; // skip first coinbase (6 outputs)
+  size_t rct_idx = 0;
+  size_t pre_rct_idx = 0;
+  for (size_t out_idx_idx = 0; out_idx[out_idx_idx] >= 0; ++out_idx_idx) {
+    sources.resize(sources.size()+1);
+    tx_source_entry& src = sources.back();
+
+    src.real_output = 0;
+    if (out_idx[out_idx_idx]) {
+      // rct
+      src.amount = 7390000000000;
+      src.real_out_tx_key = get_tx_pub_key_from_extra(rct_txes[rct_idx/4]);
+      src.real_output_in_tx_index = rct_idx&3;
+      src.mask = rct_tx_masks[rct_idx];
+      src.rct = true;
+      for (int m = 0; m <= mixin; ++m) {
+        rct::ctkey ctkey;
+        ctkey.dest = rct::pk2rct(boost::get<txout_to_key>(rct_txes[rct_idx/4].vout[rct_idx&3].target).key);
+        ctkey.mask = rct_txes[rct_idx/4].rct_signatures.outPk[rct_idx&3].mask;
+        src.outputs.push_back(std::make_pair(global_rct_idx, ctkey));
+        ++rct_idx;
+        ++global_rct_idx;
+        if (global_rct_idx % 10 == 0)
+          global_rct_idx += 6; // skip the coinbase
+      }
+    }
+    else
+    {
+      // pre rct
+      src.amount = 5000000000000;
+      src.real_out_tx_key = cryptonote::get_tx_pub_key_from_extra(blocks[pre_rct_idx].miner_tx);
+      src.real_output_in_tx_index = 4;
+      src.mask = rct::identity();
+      src.rct = false;
+      for (int m = 0; m <= mixin; ++m) {
+        src.push_output(m, boost::get<txout_to_key>(blocks[pre_rct_idx].miner_tx.vout[4].target).key, src.amount);
+        ++pre_rct_idx;
+      }
+    }
+  }
+
+  //fill outputs entry
+  tx_destination_entry td;
+  td.addr = miner_account.get_keys().m_account_address;
+  td.amount = amount_paid;
+  std::vector<tx_destination_entry> destinations;
+  destinations.push_back(td);
+
+  if (pre_tx)
+    pre_tx(sources, destinations);
+
+  transaction tx;
+  crypto::secret_key tx_key;
+  bool r = construct_tx_and_get_tx_key(miner_accounts[0].get_keys(), sources, destinations, std::vector<uint8_t>(), tx, 0, tx_key, true);
+  CHECK_AND_ASSERT_MES(r, false, "failed to construct transaction");
+
+  if (post_tx)
+    post_tx(tx);
+
+  if (!valid)
+    DO_CALLBACK(events, "mark_invalid_tx");
+  events.push_back(tx);
+  LOG_PRINT_L0("Test tx: " << obj_to_json_str(tx));
+
+  return true;
+}
+
+bool gen_rct_tx_valid_from_pre_rct::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, true, NULL, NULL);
+}
+
+bool gen_rct_tx_valid_from_rct::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, true, NULL, NULL);
+}
+
+bool gen_rct_tx_valid_from_mixed::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, 0, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, true, NULL, NULL);
+}
+
+bool gen_rct_tx_pre_rct_bad_real_dest::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  bool tx_creation_succeeded = false;
+  // in the case, the tx will fail to create, due to mismatched sk/pk
+  bool ret = generate_with(events, out_idx, mixin, amount_paid, false,
+    [](std::vector<tx_source_entry> &sources, std::vector<tx_destination_entry> &destinations) {rct::key sk; rct::skpkGen(sk, sources[0].outputs[0].second.dest);},
+    [&tx_creation_succeeded](const transaction &tx){tx_creation_succeeded=true;});
+  return !ret && !tx_creation_succeeded;
+}
+
+bool gen_rct_tx_pre_rct_bad_real_mask::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    [](std::vector<tx_source_entry> &sources, std::vector<tx_destination_entry> &destinations) {sources[0].outputs[0].second.mask = rct::zeroCommit(99999);},
+    NULL);
+}
+
+bool gen_rct_tx_pre_rct_bad_fake_dest::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    [](std::vector<tx_source_entry> &sources, std::vector<tx_destination_entry> &destinations) {rct::key sk; rct::skpkGen(sk, sources[0].outputs[1].second.dest);},
+    NULL);
+}
+
+bool gen_rct_tx_pre_rct_bad_fake_mask::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    [](std::vector<tx_source_entry> &sources, std::vector<tx_destination_entry> &destinations) {sources[0].outputs[1].second.mask = rct::zeroCommit(99999);},
+    NULL);
+}
+
+bool gen_rct_tx_rct_bad_real_dest::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  bool tx_creation_succeeded = false;
+  // in the case, the tx will fail to create, due to mismatched sk/pk
+  bool ret = generate_with(events, out_idx, mixin, amount_paid, false,
+    [](std::vector<tx_source_entry> &sources, std::vector<tx_destination_entry> &destinations) {rct::key sk; rct::skpkGen(sk, sources[0].outputs[0].second.dest);},
+    [&tx_creation_succeeded](const transaction &tx){tx_creation_succeeded=true;});
+  return !ret && !tx_creation_succeeded;
+}
+
+bool gen_rct_tx_rct_bad_real_mask::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    [](std::vector<tx_source_entry> &sources, std::vector<tx_destination_entry> &destinations) {sources[0].outputs[0].second.mask = rct::zeroCommit(99999);},
+    NULL);
+}
+
+bool gen_rct_tx_rct_bad_fake_dest::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    [](std::vector<tx_source_entry> &sources, std::vector<tx_destination_entry> &destinations) {rct::key sk; rct::skpkGen(sk, sources[0].outputs[1].second.dest);},
+    NULL);
+}
+
+bool gen_rct_tx_rct_bad_fake_mask::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    [](std::vector<tx_source_entry> &sources, std::vector<tx_destination_entry> &destinations) {sources[0].outputs[1].second.mask = rct::zeroCommit(99999);},
+    NULL);
+}
+
+bool gen_rct_tx_rct_spend_with_zero_commit::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    [](std::vector<tx_source_entry> &sources, std::vector<tx_destination_entry> &destinations) {sources[0].outputs[0].second.mask = rct::zeroCommit(sources[0].amount); sources[0].mask = rct::identity();},
+    [](transaction &tx){boost::get<txin_to_key>(tx.vin[0]).amount = 0;});
+}
+
+bool gen_rct_tx_pre_rct_zero_vin_amount::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {boost::get<txin_to_key>(tx.vin[0]).amount = 0;});
+}
+
+bool gen_rct_tx_rct_non_zero_vin_amount::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {boost::get<txin_to_key>(tx.vin[0]).amount = 5000000000000;}); // one that we know exists
+}
+
+bool gen_rct_tx_non_zero_vout_amount::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {tx.vout[0].amount = 5000000000000;}); // one that we know exists
+}
+
+bool gen_rct_tx_pre_rct_duplicate_key_image::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [&events](transaction &tx) {boost::get<txin_to_key>(tx.vin[0]).k_image = boost::get<txin_to_key>(boost::get<transaction>(events[67]).vin[0]).k_image;});
+}
+
+bool gen_rct_tx_rct_duplicate_key_image::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [&events](transaction &tx) {boost::get<txin_to_key>(tx.vin[0]).k_image = boost::get<txin_to_key>(boost::get<transaction>(events[67]).vin[0]).k_image;});
+}
+
+bool gen_rct_tx_pre_rct_wrong_key_image::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  // some random key image from the monero blockchain, so we get something that is a valid key image
+  static const uint8_t k_image[33] = "\x49\x3b\x56\x16\x54\x76\xa8\x75\xb7\xf4\xa8\x51\xf5\x55\xd3\x44\xe7\x3e\xea\x73\xee\xc1\x06\x7c\x7d\xb6\x57\x28\x46\x85\xe1\x07";
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {memcpy(&boost::get<txin_to_key>(tx.vin[0]).k_image, k_image, 32);});
+}
+
+bool gen_rct_tx_rct_wrong_key_image::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  // some random key image from the monero blockchain, so we get something that is a valid key image
+  static const uint8_t k_image[33] = "\x49\x3b\x56\x16\x54\x76\xa8\x75\xb7\xf4\xa8\x51\xf5\x55\xd3\x44\xe7\x3e\xea\x73\xee\xc1\x06\x7c\x7d\xb6\x57\x28\x46\x85\xe1\x07";
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {memcpy(&boost::get<txin_to_key>(tx.vin[0]).k_image, k_image, 32);});
+}
+
+bool gen_rct_tx_pre_rct_wrong_fee::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {tx.rct_signatures.txnFee++;});
+}
+
+bool gen_rct_tx_rct_wrong_fee::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {tx.rct_signatures.txnFee++;});
+}
+
+bool gen_rct_tx_pre_rct_increase_vin_and_fee::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {boost::get<txin_to_key>(tx.vin[0]).amount++;tx.rct_signatures.txnFee++;});
+}
+
+bool gen_rct_tx_pre_rct_remove_vin::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {tx.vin.pop_back();});
+}
+
+bool gen_rct_tx_rct_remove_vin::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {tx.vin.pop_back();});
+}
+
+bool gen_rct_tx_pre_rct_add_vout::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {tx.vout.push_back(tx.vout.back());});
+}
+
+bool gen_rct_tx_rct_add_vout::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {tx.vout.push_back(tx.vout.back());});
+}
+
+bool gen_rct_tx_pre_rct_altered_extra::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {0, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {std::string extra_nonce; crypto::hash pid = cryptonote::null_hash; set_payment_id_to_tx_extra_nonce(extra_nonce, pid); add_extra_nonce_to_tx_extra(tx.extra, extra_nonce);});
+}
+
+bool gen_rct_tx_rct_altered_extra::generate(std::vector<test_event_entry>& events) const
+{
+  const int mixin = 2;
+  const int out_idx[] = {1, -1};
+  const uint64_t amount_paid = 10000;
+  return generate_with(events, out_idx, mixin, amount_paid, false,
+    NULL, [](transaction &tx) {std::string extra_nonce; crypto::hash pid = cryptonote::null_hash; set_payment_id_to_tx_extra_nonce(extra_nonce, pid); add_extra_nonce_to_tx_extra(tx.extra, extra_nonce);});
+}
+

tests/core_tests/rct.h

@@ -0,0 +1,264 @@
+// Copyright (c) 2014-2016, The Monero Project
+// 
+// All rights reserved.
+// 
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+// 
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+// 
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+// 
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+// 
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+// 
+// Parts of this file are originally copyright (c) 2012-2013 The Cryptonote developers
+
+#pragma once 
+#include "chaingen.h"
+
+struct gen_rct_tx_validation_base : public test_chain_unit_base
+{
+  gen_rct_tx_validation_base()
+    : m_invalid_tx_index(0)
+    , m_invalid_block_index(0)
+  {
+    REGISTER_CALLBACK_METHOD(gen_rct_tx_validation_base, mark_invalid_tx);
+    REGISTER_CALLBACK_METHOD(gen_rct_tx_validation_base, mark_invalid_block);
+  }
+
+  bool check_tx_verification_context(const cryptonote::tx_verification_context& tvc, bool tx_added, size_t event_idx, const cryptonote::transaction& /*tx*/)
+  {
+    if (m_invalid_tx_index == event_idx)
+      return tvc.m_verifivation_failed;
+    else
+      return !tvc.m_verifivation_failed && tx_added;
+  }
+
+  bool check_block_verification_context(const cryptonote::block_verification_context& bvc, size_t event_idx, const cryptonote::block& /*block*/)
+  {
+    if (m_invalid_block_index == event_idx)
+      return bvc.m_verifivation_failed;
+    else
+      return !bvc.m_verifivation_failed;
+  }
+
+  bool mark_invalid_block(cryptonote::core& /*c*/, size_t ev_index, const std::vector<test_event_entry>& /*events*/)
+  {
+    m_invalid_block_index = ev_index + 1;
+    return true;
+  }
+
+  bool mark_invalid_tx(cryptonote::core& /*c*/, size_t ev_index, const std::vector<test_event_entry>& /*events*/)
+  {
+    m_invalid_tx_index = ev_index + 1;
+    return true;
+  }
+
+  bool generate_with(std::vector<test_event_entry>& events, const int *out_idx, int mixin,
+      uint64_t amount_paid, bool valid,
+      const std::function<void(std::vector<cryptonote::tx_source_entry> &sources, std::vector<cryptonote::tx_destination_entry> &destinations)> &pre_tx,
+      const std::function<void(cryptonote::transaction &tx)> &post_tx) const;
+
+private:
+  size_t m_invalid_tx_index;
+  size_t m_invalid_block_index;
+};
+
+template<>
+struct get_test_options<gen_rct_tx_validation_base> {
+  const std::pair<uint8_t, uint64_t> hard_forks[4] = {std::make_pair(1, 0), std::make_pair(2, 1), std::make_pair(4, 65), std::make_pair(0, 0)};
+  const cryptonote::test_options test_options = {
+    hard_forks
+  };
+};
+
+// valid
+struct gen_rct_tx_valid_from_pre_rct : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_valid_from_pre_rct>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_valid_from_rct : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_valid_from_rct>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_valid_from_mixed : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_valid_from_mixed>: public get_test_options<gen_rct_tx_validation_base> {};
+
+// altered commitment/dest
+struct gen_rct_tx_pre_rct_bad_real_dest : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_pre_rct_bad_real_dest>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_pre_rct_bad_real_mask : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_pre_rct_bad_real_mask>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_pre_rct_bad_fake_dest : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_pre_rct_bad_fake_dest>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_pre_rct_bad_fake_mask : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_pre_rct_bad_fake_mask>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_rct_bad_real_dest : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_rct_bad_real_dest>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_rct_bad_real_mask : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_rct_bad_real_mask>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_rct_bad_fake_dest : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_rct_bad_fake_dest>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_rct_bad_fake_mask : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_rct_bad_fake_mask>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_rct_spend_with_zero_commit : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_rct_spend_with_zero_commit>: public get_test_options<gen_rct_tx_validation_base> {};
+
+// altered amounts
+struct gen_rct_tx_pre_rct_zero_vin_amount : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_pre_rct_zero_vin_amount>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_rct_non_zero_vin_amount : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_rct_non_zero_vin_amount>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_non_zero_vout_amount : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_non_zero_vout_amount>: public get_test_options<gen_rct_tx_validation_base> {};
+
+// key image
+struct gen_rct_tx_pre_rct_duplicate_key_image : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_pre_rct_duplicate_key_image>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_rct_duplicate_key_image : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_rct_duplicate_key_image>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_pre_rct_wrong_key_image : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_pre_rct_wrong_key_image>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_rct_wrong_key_image : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_rct_wrong_key_image>: public get_test_options<gen_rct_tx_validation_base> {};
+
+// fee
+struct gen_rct_tx_pre_rct_wrong_fee : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_pre_rct_wrong_fee>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_rct_wrong_fee : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_rct_wrong_fee>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_pre_rct_increase_vin_and_fee : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_pre_rct_increase_vin_and_fee>: public get_test_options<gen_rct_tx_validation_base> {};
+
+// modify vin/vout
+struct gen_rct_tx_pre_rct_remove_vin : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_pre_rct_remove_vin>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_rct_remove_vin : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_rct_remove_vin>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_pre_rct_add_vout : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_pre_rct_add_vout>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_rct_add_vout : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_rct_add_vout>: public get_test_options<gen_rct_tx_validation_base> {};
+
+// extra
+struct gen_rct_tx_pre_rct_altered_extra : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_pre_rct_altered_extra>: public get_test_options<gen_rct_tx_validation_base> {};
+
+struct gen_rct_tx_rct_altered_extra : public gen_rct_tx_validation_base
+{
+  bool generate(std::vector<test_event_entry>& events) const;
+};
+template<> struct get_test_options<gen_rct_tx_rct_altered_extra>: public get_test_options<gen_rct_tx_validation_base> {};
+

tests/core_tests/transaction_tests.cpp

@@ -82,32 +82,22 @@ bool test_transaction_generation_and_ring_signature()
   src.amount = 70368744177663;
   {
     tx_output_entry oe;
-    oe.first = 0;
-    oe.second = boost::get<txout_to_key>(tx_mine_1.vout[0].target).key;
-    src.outputs.push_back(oe);
 
-    oe.first = 1;
+    src.push_output(0, boost::get<txout_to_key>(tx_mine_1.vout[0].target).key, src.amount);
-    oe.second = boost::get<txout_to_key>(tx_mine_2.vout[0].target).key;
-    src.outputs.push_back(oe);
 
-    oe.first = 2;
+    src.push_output(1, boost::get<txout_to_key>(tx_mine_2.vout[0].target).key, src.amount);
-    oe.second = boost::get<txout_to_key>(tx_mine_3.vout[0].target).key;
-    src.outputs.push_back(oe);
 
-    oe.first = 3;
+    src.push_output(2, boost::get<txout_to_key>(tx_mine_3.vout[0].target).key, src.amount);
-    oe.second = boost::get<txout_to_key>(tx_mine_4.vout[0].target).key;
-    src.outputs.push_back(oe);
 
-    oe.first = 4;
+    src.push_output(3, boost::get<txout_to_key>(tx_mine_4.vout[0].target).key, src.amount);
-    oe.second = boost::get<txout_to_key>(tx_mine_5.vout[0].target).key;
-    src.outputs.push_back(oe);
 
-    oe.first = 5;
+    src.push_output(4, boost::get<txout_to_key>(tx_mine_5.vout[0].target).key, src.amount);
-    oe.second = boost::get<txout_to_key>(tx_mine_6.vout[0].target).key;
+
-    src.outputs.push_back(oe);
+    src.push_output(5, boost::get<txout_to_key>(tx_mine_6.vout[0].target).key, src.amount);
 
     src.real_out_tx_key = cryptonote::get_tx_pub_key_from_extra(tx_mine_2);
     src.real_output = 1;
+    src.rct = false;
     src.real_output_in_tx_index = 0;
   }
   //fill outputs entry

tests/core_tests/tx_validation.cpp

@@ -39,7 +39,7 @@ namespace
 {
   struct tx_builder
   {
-    void step1_init(size_t version = CURRENT_TRANSACTION_VERSION, uint64_t unlock_time = 0)
+    void step1_init(size_t version = 1, uint64_t unlock_time = 0)
     {
       m_tx.vin.clear();
       m_tx.vout.clear();
@@ -108,9 +108,13 @@ namespace
       BOOST_FOREACH(const tx_source_entry& src_entr, sources)
       {
         std::vector<const crypto::public_key*> keys_ptrs;
+        std::vector<crypto::public_key> keys(src_entr.outputs.size());
+        size_t j = 0;
         BOOST_FOREACH(const tx_source_entry::output_entry& o, src_entr.outputs)
         {
-          keys_ptrs.push_back(&o.second);
+          keys[j] = rct::rct2pk(o.second.dest);
+          keys_ptrs.push_back(&keys[j]);
+          ++j;
         }
 
         m_tx.signatures.push_back(std::vector<crypto::signature>());
@@ -136,7 +140,7 @@ namespace
     fill_tx_sources_and_destinations(events, blk_head, from, to, amount, TESTS_DEFAULT_FEE, 0, sources, destinations);
 
     tx_builder builder;
-    builder.step1_init(CURRENT_TRANSACTION_VERSION, unlock_time);
+    builder.step1_init(1, unlock_time);
     builder.step2_fill_inputs(from.get_keys(), sources);
     builder.step3_fill_outputs(destinations);
     builder.step4_calc_hash();
@@ -177,7 +181,7 @@ bool gen_tx_big_version::generate(std::vector<test_event_entry>& events) const
   fill_tx_sources_and_destinations(events, blk_0, miner_account, miner_account, MK_COINS(1), TESTS_DEFAULT_FEE, 0, sources, destinations);
 
   tx_builder builder;
-  builder.step1_init(CURRENT_TRANSACTION_VERSION + 1, 0);
+  builder.step1_init(1 + 1, 0);
   builder.step2_fill_inputs(miner_account.get_keys(), sources);
   builder.step3_fill_outputs(destinations);
   builder.step4_calc_hash();

tests/core_tests/v2_tests.cpp

@@ -79,7 +79,6 @@ bool gen_v2_tx_validation_base::generate_with(std::vector<test_event_entry>& eve
   }
 
   // create a tx with the Nth outputs of miner's block reward
-  typedef tx_source_entry::output_entry tx_output_entry;
   std::vector<tx_source_entry> sources;
   for (size_t out_idx_idx = 0; out_idx[out_idx_idx] >= 0; ++out_idx_idx) {
     sources.resize(sources.size()+1);
@@ -88,16 +87,16 @@ bool gen_v2_tx_validation_base::generate_with(std::vector<test_event_entry>& eve
     src.amount = blocks[0].miner_tx.vout[out_idx[out_idx_idx]].amount;
   std::cout << "using " << print_money(src.amount) << " output at index " << out_idx[out_idx_idx] << std::endl;
     for (int m = 0; m <= mixin; ++m) {
-      tx_output_entry oe;
+      int idx;
       if (is_valid_decomposed_amount(src.amount))
-        oe.first = m+1; // one out of that size per miner tx, including genesis
+        idx = m+1; // one out of that size per miner tx, including genesis
       else
-        oe.first = 0; // dusty, no other output of that size
+        idx = 0; // dusty, no other output of that size
-      oe.second = boost::get<txout_to_key>(blocks[m].miner_tx.vout[out_idx[out_idx_idx]].target).key;
+      src.push_output(idx, boost::get<txout_to_key>(blocks[m].miner_tx.vout[out_idx[out_idx_idx]].target).key, src.amount);
-      src.outputs.push_back(oe);
     }
     src.real_out_tx_key = cryptonote::get_tx_pub_key_from_extra(blocks[0].miner_tx);
     src.real_output = 0;
+    src.rct = false;
     src.real_output_in_tx_index = out_idx[out_idx_idx];
   }
 

tests/core_tests/v2_tests.h

@@ -79,7 +79,7 @@ private:
 
 template<>
 struct get_test_options<gen_v2_tx_validation_base> {
-  const std::pair<uint8_t, uint64_t> hard_forks[2] = {std::make_pair(1, 0), std::make_pair(2, 1)};
+  const std::pair<uint8_t, uint64_t> hard_forks[3] = {std::make_pair(1, 0), std::make_pair(2, 1), std::make_pair(0, 0)};
   const cryptonote::test_options test_options = {
     hard_forks
   };

tests/functional_tests/transactions_flow_test.cpp

@@ -281,7 +281,7 @@ bool transactions_flow_test(std::string& working_folder,
     w2.get_transfers(tc);
     BOOST_FOREACH(tools::wallet2::transfer_details& td, tc)
     {
-      auto it = txs.find(get_transaction_hash(td.m_tx));
+      auto it = txs.find(td.m_txid);
       CHECK_AND_ASSERT_MES(it != txs.end(), false, "transaction not found in local cache");
       it->second.m_received_count += 1;
     }

tests/functional_tests/transactions_generation_from_blockchain.cpp

@@ -31,7 +31,6 @@
 #include "include_base_utils.h"
 using namespace epee;
 #include "wallet/wallet2.h"
-#include "cryptonote_core/blockchain_storage.h"
 
 using namespace cryptonote;
 
@@ -114,6 +113,7 @@ bool make_tx(blockchain_storage& bch)
     src.real_out_tx_key = td.m_tx.tx_pub_key;
     src.real_output = interted_it - src.outputs.begin();
     src.real_output_in_tx_index = td.m_internal_output_index;
+    src.rct = false;
     ++i;
   }
 

tests/performance_tests/multi_tx_test_base.h

@@ -59,7 +59,7 @@ public:
         return false;
 
       txout_to_key tx_out = boost::get<txout_to_key>(m_miner_txs[i].vout[0].target);
-      output_entries.push_back(std::make_pair(i, tx_out.key));
+      output_entries.push_back(std::make_pair(i, rct::ctkey({rct::pk2rct(tx_out.key), rct::identity()})));
       m_public_keys[i] = tx_out.key;
       m_public_key_ptrs[i] = &m_public_keys[i];
     }
@@ -72,6 +72,7 @@ public:
     source_entry.real_output_in_tx_index = 0;
     source_entry.outputs.swap(output_entries);
     source_entry.real_output = real_source_idx;
+    source_entry.rct = false;
 
     m_sources.push_back(source_entry);
 

tests/unit_tests/CMakeLists.txt

@@ -51,7 +51,9 @@ set(unit_tests_sources
   test_protocol_pack.cpp
   hardfork.cpp
   unbound.cpp
-  varint.cpp)
+  varint.cpp
+  ringct.cpp
+  output_selection.cpp)
 
 set(unit_tests_headers
   unit_tests_utils.h)
@@ -61,6 +63,7 @@ add_executable(unit_tests
   ${unit_tests_headers})
 target_link_libraries(unit_tests
   LINK_PRIVATE
+    ringct
     cryptonote_core
     blockchain_db
     rpc

tests/unit_tests/hardfork.cpp

@@ -96,7 +96,7 @@ public:
   virtual void remove_block() { blocks.pop_back(); }
   virtual uint64_t add_transaction_data(const crypto::hash& blk_hash, const transaction& tx, const crypto::hash& tx_hash) {return 0;}
   virtual void remove_transaction_data(const crypto::hash& tx_hash, const transaction& tx) {}
-  virtual uint64_t add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index, const uint64_t unlock_time) {return 0;}
+  virtual uint64_t add_output(const crypto::hash& tx_hash, const tx_out& tx_output, const uint64_t& local_index, const uint64_t unlock_time, const rct::key *commitment) {return 0;}
   virtual void add_tx_amount_output_indices(const uint64_t tx_index, const std::vector<uint64_t>& amount_output_indices) {}
   virtual void add_spent_key(const crypto::key_image& k_image) {}
   virtual void remove_spent_key(const crypto::key_image& k_image) {}

tests/unit_tests/output_selection.cpp

@@ -0,0 +1,103 @@
+// Copyright (c) 2014-2016, The Monero Project
+// 
+// All rights reserved.
+// 
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+// 
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+// 
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+// 
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+// 
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+// FIXME: move this into a full wallet2 unit test suite, if possible
+
+#include "gtest/gtest.h"
+
+#include "wallet/wallet2.h"
+#include <string>
+
+static tools::wallet2::transfer_container make_transfers_container(size_t N)
+{
+  tools::wallet2::transfer_container transfers;
+  for (size_t n = 0; n < N; ++n)
+  {
+    transfers.push_back(AUTO_VAL_INIT(tools::wallet2::transfer_details()));
+    tools::wallet2::transfer_details &td = transfers.back();
+    td.m_block_height = 1000;
+    td.m_spent = false;
+    td.m_txid = cryptonote::null_hash;
+    td.m_txid.data[0] = n & 0xff;
+    td.m_txid.data[1] = (n >> 8) & 0xff;
+    td.m_txid.data[2] = (n >> 16) & 0xff;
+    td.m_txid.data[3] = (n >> 24) & 0xff;
+  }
+  return transfers;
+}
+
+#define SELECT(idx) \
+  do { \
+    auto i = std::find(unused_indices.begin(), unused_indices.end(), idx); \
+    ASSERT_TRUE(i != unused_indices.end()); \
+    unused_indices.erase(i); \
+    selected.push_back(transfers.begin() + idx); \
+  } while(0)
+
+#define PICK(expected) \
+  do { \
+    size_t idx = w.pop_best_value_from(transfers, unused_indices, selected); \
+    ASSERT_EQ(expected, idx); \
+    selected.push_back(transfers.begin() + idx); \
+  } while(0)
+
+TEST(select_outputs, one_out_of_N)
+{
+  tools::wallet2 w;
+
+  // check that if there are N-1 outputs of the same height, one of them
+  // already selected, the next one selected is the one that's from a
+  // different height
+  tools::wallet2::transfer_container transfers = make_transfers_container(10);
+  transfers[6].m_block_height = 700;
+  std::vector<size_t> unused_indices({0, 1, 2, 3, 4, 5, 6, 7, 8, 9});
+  std::list<tools::wallet2::transfer_container::iterator> selected;
+  SELECT(2);
+  PICK(6);
+}
+
+TEST(select_outputs, order)
+{
+  tools::wallet2 w;
+
+  // check that most unrelated heights are picked in order
+  tools::wallet2::transfer_container transfers = make_transfers_container(5);
+  transfers[0].m_block_height = 700;
+  transfers[1].m_block_height = 700;
+  transfers[2].m_block_height = 704;
+  transfers[3].m_block_height = 716;
+  transfers[4].m_block_height = 701;
+  std::vector<size_t> unused_indices({0, 1, 2, 3, 4});
+  std::list<tools::wallet2::transfer_container::iterator> selected;
+  SELECT(0);
+  PICK(3); // first the one that's far away
+  PICK(2); // then the one that's close
+  PICK(4); // then the one that's adjacent
+  PICK(1); // then the one that's on the same height
+}
+

tests/unit_tests/serialization.cpp

@@ -36,6 +36,7 @@
 #include <boost/foreach.hpp>
 #include "cryptonote_core/cryptonote_basic.h"
 #include "cryptonote_core/cryptonote_basic_impl.h"
+#include "ringct/rctSigs.h"
 #include "serialization/serialization.h"
 #include "serialization/binary_archive.h"
 #include "serialization/json_archive.h"
@@ -442,3 +443,200 @@ TEST(Serialization, serializes_transacion_signatures_correctly)
   blob.resize(blob.size() + sizeof(crypto::signature) / 2);
   ASSERT_FALSE(serialization::parse_binary(blob, tx1));
 }
+
+TEST(Serialization, serializes_ringct_types)
+{
+  string blob;
+  rct::key key0, key1;
+  rct::keyV keyv0, keyv1;
+  rct::keyM keym0, keym1;
+  rct::ctkey ctkey0, ctkey1;
+  rct::ctkeyV ctkeyv0, ctkeyv1;
+  rct::ctkeyM ctkeym0, ctkeym1;
+  rct::ecdhTuple ecdh0, ecdh1;
+  rct::asnlSig asnl0, asnl1;
+  rct::mgSig mg0, mg1;
+  rct::rangeSig rg0, rg1;
+  rct::rctSig s0, s1;
+  cryptonote::transaction tx0, tx1;
+
+  key0 = rct::skGen();
+  ASSERT_TRUE(serialization::dump_binary(key0, blob));
+  ASSERT_TRUE(serialization::parse_binary(blob, key1));
+  ASSERT_TRUE(key0 == key1);
+
+  keyv0 = rct::skvGen(30);
+  for (size_t n = 0; n < keyv0.size(); ++n)
+    keyv0[n] = rct::skGen();
+  ASSERT_TRUE(serialization::dump_binary(keyv0, blob));
+  ASSERT_TRUE(serialization::parse_binary(blob, keyv1));
+  ASSERT_TRUE(keyv0.size() == keyv1.size());
+  for (size_t n = 0; n < keyv0.size(); ++n)
+  {
+    ASSERT_TRUE(keyv0[n] == keyv1[n]);
+  }
+
+  keym0 = rct::keyMInit(9, 12);
+  for (size_t n = 0; n < keym0.size(); ++n)
+    for (size_t i = 0; i < keym0[n].size(); ++i)
+      keym0[n][i] = rct::skGen();
+  ASSERT_TRUE(serialization::dump_binary(keym0, blob));
+  ASSERT_TRUE(serialization::parse_binary(blob, keym1));
+  ASSERT_TRUE(keym0.size() == keym1.size());
+  for (size_t n = 0; n < keym0.size(); ++n)
+  {
+    ASSERT_TRUE(keym0[n].size() == keym1[n].size());
+    for (size_t i = 0; i < keym0[n].size(); ++i)
+    {
+      ASSERT_TRUE(keym0[n][i] == keym1[n][i]);
+    }
+  }
+
+  rct::skpkGen(ctkey0.dest, ctkey0.mask);
+  ASSERT_TRUE(serialization::dump_binary(ctkey0, blob));
+  ASSERT_TRUE(serialization::parse_binary(blob, ctkey1));
+  ASSERT_TRUE(!memcmp(&ctkey0, &ctkey1, sizeof(ctkey0)));
+
+  ctkeyv0 = std::vector<rct::ctkey>(14);
+  for (size_t n = 0; n < ctkeyv0.size(); ++n)
+    rct::skpkGen(ctkeyv0[n].dest, ctkeyv0[n].mask);
+  ASSERT_TRUE(serialization::dump_binary(ctkeyv0, blob));
+  ASSERT_TRUE(serialization::parse_binary(blob, ctkeyv1));
+  ASSERT_TRUE(ctkeyv0.size() == ctkeyv1.size());
+  for (size_t n = 0; n < ctkeyv0.size(); ++n)
+  {
+    ASSERT_TRUE(!memcmp(&ctkeyv0[n], &ctkeyv1[n], sizeof(ctkeyv0[n])));
+  }
+
+  ctkeym0 = std::vector<rct::ctkeyV>(9);
+  for (size_t n = 0; n < ctkeym0.size(); ++n)
+  {
+    ctkeym0[n] = std::vector<rct::ctkey>(11);
+    for (size_t i = 0; i < ctkeym0[n].size(); ++i)
+      rct::skpkGen(ctkeym0[n][i].dest, ctkeym0[n][i].mask);
+  }
+  ASSERT_TRUE(serialization::dump_binary(ctkeym0, blob));
+  ASSERT_TRUE(serialization::parse_binary(blob, ctkeym1));
+  ASSERT_TRUE(ctkeym0.size() == ctkeym1.size());
+  for (size_t n = 0; n < ctkeym0.size(); ++n)
+  {
+    ASSERT_TRUE(ctkeym0[n].size() == ctkeym1[n].size());
+    for (size_t i = 0; i < ctkeym0.size(); ++i)
+    {
+      ASSERT_TRUE(!memcmp(&ctkeym0[n][i], &ctkeym1[n][i], sizeof(ctkeym0[n][i])));
+    }
+  }
+
+  ecdh0.mask = rct::skGen();
+  ecdh0.amount = rct::skGen();
+  ecdh0.senderPk = rct::skGen();
+  ASSERT_TRUE(serialization::dump_binary(ecdh0, blob));
+  ASSERT_TRUE(serialization::parse_binary(blob, ecdh1));
+  ASSERT_TRUE(!memcmp(&ecdh0.mask, &ecdh1.mask, sizeof(ecdh0.mask)));
+  ASSERT_TRUE(!memcmp(&ecdh0.amount, &ecdh1.amount, sizeof(ecdh0.amount)));
+  // senderPk is not serialized
+
+  for (size_t n = 0; n < 64; ++n)
+  {
+    asnl0.L1[n] = rct::skGen();
+    asnl0.s2[n] = rct::skGen();
+  }
+  asnl0.s = rct::skGen();
+  ASSERT_TRUE(serialization::dump_binary(asnl0, blob));
+  ASSERT_TRUE(serialization::parse_binary(blob, asnl1));
+  ASSERT_TRUE(!memcmp(&asnl0, &asnl1, sizeof(asnl0)));
+
+  // create a full rct signature to use its innards
+  rct::ctkeyV sc, pc;
+  rct::ctkey sctmp, pctmp;
+  tie(sctmp, pctmp) = rct::ctskpkGen(6000);
+  sc.push_back(sctmp);
+  pc.push_back(pctmp);
+  tie(sctmp, pctmp) = rct::ctskpkGen(7000);
+  sc.push_back(sctmp);
+  pc.push_back(pctmp);
+  vector<uint64_t> amounts;
+  rct::keyV amount_keys;
+  //add output 500
+  amounts.push_back(500);
+  rct::keyV destinations;
+  rct::key Sk, Pk;
+  rct::skpkGen(Sk, Pk);
+  destinations.push_back(Pk);
+  //add output for 12500
+  amounts.push_back(12500);
+  amount_keys.push_back(rct::hash_to_scalar(rct::zero()));
+  rct::skpkGen(Sk, Pk);
+  destinations.push_back(Pk);
+  //compute rct data with mixin 500
+  s0 = rct::genRct(rct::zero(), sc, pc, destinations, amounts, amount_keys, 3);
+
+  mg0 = s0.p.MGs[0];
+  ASSERT_TRUE(serialization::dump_binary(mg0, blob));
+  ASSERT_TRUE(serialization::parse_binary(blob, mg1));
+  ASSERT_TRUE(mg0.ss.size() == mg1.ss.size());
+  for (size_t n = 0; n < mg0.ss.size(); ++n)
+  {
+    ASSERT_TRUE(mg0.ss[n] == mg1.ss[n]);
+  }
+  ASSERT_TRUE(mg0.cc == mg1.cc);
+
+  // mixRing and II are not serialized, they are meant to be reconstructed
+  ASSERT_TRUE(mg1.II.empty());
+
+  rg0 = s0.p.rangeSigs.front();
+  ASSERT_TRUE(serialization::dump_binary(rg0, blob));
+  ASSERT_TRUE(serialization::parse_binary(blob, rg1));
+  ASSERT_TRUE(!memcmp(&rg0, &rg1, sizeof(rg0)));
+
+  ASSERT_TRUE(serialization::dump_binary(s0, blob));
+  ASSERT_TRUE(serialization::parse_binary(blob, s1));
+  ASSERT_TRUE(s0.type == s1.type);
+  ASSERT_TRUE(s0.p.rangeSigs.size() == s1.p.rangeSigs.size());
+  for (size_t n = 0; n < s0.p.rangeSigs.size(); ++n)
+  {
+    ASSERT_TRUE(!memcmp(&s0.p.rangeSigs[n], &s1.p.rangeSigs[n], sizeof(s0.p.rangeSigs[n])));
+  }
+  ASSERT_TRUE(s0.p.MGs.size() == s1.p.MGs.size());
+  ASSERT_TRUE(s0.p.MGs[0].ss.size() == s1.p.MGs[0].ss.size());
+  for (size_t n = 0; n < s0.p.MGs[0].ss.size(); ++n)
+  {
+    ASSERT_TRUE(s0.p.MGs[0].ss[n] == s1.p.MGs[0].ss[n]);
+  }
+  ASSERT_TRUE(s0.p.MGs[0].cc == s1.p.MGs[0].cc);
+  // mixRing and II are not serialized, they are meant to be reconstructed
+  ASSERT_TRUE(s1.p.MGs[0].II.empty());
+
+  // mixRing and II are not serialized, they are meant to be reconstructed
+  ASSERT_TRUE(s1.mixRing.size() == 0);
+
+  ASSERT_TRUE(s0.ecdhInfo.size() == s1.ecdhInfo.size());
+  for (size_t n = 0; n < s0.ecdhInfo.size(); ++n)
+  {
+    ASSERT_TRUE(!memcmp(&s0.ecdhInfo[n], &s1.ecdhInfo[n], sizeof(s0.ecdhInfo[n])));
+  }
+  ASSERT_TRUE(s0.outPk.size() == s1.outPk.size());
+  for (size_t n = 0; n < s0.outPk.size(); ++n)
+  {
+    // serialization only does the mask
+    ASSERT_TRUE(!memcmp(&s0.outPk[n].mask, &s1.outPk[n].mask, sizeof(s0.outPk[n].mask)));
+  }
+
+  tx0.set_null();
+  tx0.version = 2;
+  cryptonote::txin_to_key txin_to_key1;
+  txin_to_key1.key_offsets.resize(2);
+  cryptonote::txin_to_key txin_to_key2;
+  txin_to_key2.key_offsets.resize(2);
+  tx0.vin.push_back(txin_to_key1);
+  tx0.vin.push_back(txin_to_key2);
+  tx0.vout.push_back(cryptonote::tx_out());
+  tx0.rct_signatures = s0;
+  ASSERT_EQ(tx0.rct_signatures.p.rangeSigs.size(), 2);
+  ASSERT_TRUE(serialization::dump_binary(tx0, blob));
+  ASSERT_TRUE(serialization::parse_binary(blob, tx1));
+  ASSERT_EQ(tx1.rct_signatures.p.rangeSigs.size(), 2);
+  std::string blob2;
+  ASSERT_TRUE(serialization::dump_binary(tx1, blob2));
+  ASSERT_TRUE(blob == blob2);
+}