Grant Update December 2018

Technical Updates | January 11, 2019
Home » Company » Blog » Grant Update December 2018

Outputs and Outcomes

Our focus is to:

  • Improve the performance and reliability of the Protocol
  • Provide ability to monitor the exchange rate and make adjustments as needed
  • Make improvements to the Anchor Master allowing Factom to remain competitive and relevant in the marketplace.

Grants that expired in Dec: Protocol Grant Ph2   |   Oracle Grant Ph2   | Anchor Grant Ph2   

Factom is now operating under an interim, anticipated Continuity Grant. Details here:

https://factomize.com/forums/threads/factom-inc-inter-grant-continuity-plan-dec-feb.1423

Status and Achievements

December was a month in Factomd development dominated by bug-finding.  During development of the upcoming release, several difficult-to-find and subsequently fix bugs were identified.  Much of the debugging was related to the SaveState management. This is the section of code which allows factomd to boot quickly when started and one of the things that has been plaguing re-release of the 5.4.4 based code from July.  There were also some subtle rare bugs that caused panics that complained about negative Factoid balances. These were tricky and time consuming to debug.

There was a slight panic around the entryblockresponse message, which was initially thought to be a serve attack vector against a node, but it turned out not to be exercisable in production.

The 3 Factom, Inc sponsors continued with the 12/05 and 12/17 meetings.  The sponsors Dominic Luxford, Factomatic (Nikola), and Nolan Bauer participated with Sponsor Emeritus Nic Robinette as a spectator.  A sponsor meeting is planned to be publicly viewable in late January or February.

The Development Grant ended On Dec 10, but effort continues.  The deliverable goals laid out in the grants are planned for the next two releases.  The main deliverable of the grants is maintenance and bug fixing, and those have been the primary drives in December.  A hopeful release in January will bring several months of bug fixes into production.

WORK IN PROGRESS

New capabilities pending in next release

  • Ethereum Anchoring (Anchor Grant)
  • Standing Parties (Protocol Grant)
    • Ensures no other standing party can lay claim to the same standing
    • Cannot be intercepted by an attacker before secured in the Standing Chain
    • Can be easily tallied to calculate the standing behind a decision in the Factom protocol
  • FactomD Recognizes Voting (Protocol Grant)  
    • The tracking of identities which can be used to track the standing parties and delegate votes to provide support for ANOs, Grants, etc.
    • Provide APIs to expose information about staked FCT to allow the 3rd parties to do calculations based on this data.
    • Provide APIs to expose information about staked ECs to allow the 3rd parties to do calculations based on this data.
  • New Network Diagnostic API (Protocol Grant) – Gives visibility into aspects of nodes to better understand their performance and what could have caused a stall.
  • Rate Change Alerts (Oracle Grant) – Keeps the exchange rate current with the market to ensure they stay near the intended constant rate.
  • Sept 17th Release Feature development complete but was pulled from MainNet due to issue. Will release once issues identified and QA is complete.  
    • Performance Graphing Tools – Collecting data is complete for required for new internal graphing tools that allow QA a better understanding of how changes affect performance
    • Timeouts Defined Merged – Improvement to simulation tests so when they fail they exit
    • Calculate System Status Always – Populated Control Panel Summary\Syncing by default
    • Upgrade Golang to 1.11 – Keep up with Golang development
  • Coinbase delay for Testnet – Extend delay between a coinbase to all more time to coordinate a test of the coinbase cancel on the testnet.  
  • Diagnostic Graphs – Detailed view of the internal system performance
  • Add Simulation Tests – Increased the amount of features tested during development unit testing
  • Simulation tests in factomd_test.go should support timeouts defined in blocks – Adds ability to limit the amount of time a simulation QA test can run before declaring it has failed.
  • Message Logging Clean up – Allows the clean up of messages to better diagnose issues
  • GoLand Build Tags – GoLand does not support dynamically setting build parameters so this added a prebuild script that overwrites the engine/version.go file to accomplish what is done in production using build tags.
  • Modify factomd API call ‘current-minute’ to also return ‘current-block-height’ – Allows you to make a single API call and returns the minute and block height together to make testing more reliable
  • Upgrade to golang 1.11 – Continue standards of staying current version
  • Community Contribution for Integration of CORS code into Factomd – Allows Javascript browsers to use the data that is provided by Factomd.
  • Investigate new theory of database head reversal – This fixed a problem observed in the great stall where the last DBState in the database was not loaded when restarting after a stall which in turn caused nodes to boot to different heights which blocked the network from making forward progress without a lot of elections
  • Encrypting the Factom-walletd Database – Exchange Requests that requires password security when using wallet
  • Simulation tests in factomd_test.go should support timeouts defined in blocks – Adds ability to limit the amount of time a simulation QA test can run before declaring it has failed.
  • Move away from gopkg.in – Allows more reliable source of the source code for these packages
  • Wallet for simulator to enable more complex tests – Code refactoring to enable more complicated simulation tests involving Wallet features
  • Log Holding and write script to check data – Debugging tool to allow investigation of holding queues behavior
  • Community Feature Contribution – Show hostname on control panel – Community Contribution: Allow control panel to self identify which node it represents.
  • Make logging reopen log files that have been deleted – Allows for more detailed logging on long running diagnostics.
  • Log debugAPI – Provides diagnostic value for application developers using the debug API.

 

Maintenance

  • Continue to reduce number of missing messages – Allow the ANO nodes to communicate more effectively, have less bandwidth overhead and less risk of failure due to peer to peer network deficiencies. (Protocol Grant)
  • Sept 17th Release Development complete but was pulled from MainNet due to issue. Will release once issues identified and QA is complete.  
    • Duplicate Message reduction – Performance Improvement by no longer sending messages that were previously received.
    • MMR Improvements
      • reduced the amount of messages asked for that are not needed
      • do not ask for same message multiple times without specific time lag
      • reduce RAM consumption for process lists during reboot
    • SimCtrl EC purchase method – Improvement that allows simulation to more accurate match observed purchase behavior
    • Election Failures – Improvement to the number of false fails from simulation tests when follower/audit nodes are not keeping up
  • Out of order execution of dbstates – Corrects a bug that could cause nodes to panic.
  • Load Balance on Restore – Fixes a stall condition that causes a panic on boot
  • Nil Hash Tracking – Added code to avoid crashing in some cases with pokemon bug
  • Confile File Name Path Issue – Fixes issue where on Windows in some configurations the config files were not found.
  • Factomd sometimes writes out bad savestate files – Update the way savestate files are created to reduce consensus failures under certain conditions.
  • Invalid DBSig Handling – allows old dbsigs to be removed from process list
  • Make system Status Changes available to the control panel – This change makes the sync status of the node be available in the Control Panel for the node even if logging is off which helps diagnose stalls and other sync related issues. The changes also makes faulting and bad messages logging be on by default which helps diagnose failures of a variate of crashes by recording data about bad messages and elections.
  • Need better EC purchase method because the current one doesn’t handle high loads – This is a simulator change to make the load generated during testing behave more like the pattern observed in the real network which improves the likelihood of revealing bugs during testing.
  • WSAPI Tests were dependent on prior WSAPI test having run – Allows execution of individual test instead of just the entire suite.
  • Logging for API level transactions – Assist in debugging interactions with applications
  • ProcessBlocks failure on long-running servers after 5.4.4 release – i.e. Stop panicking when X happens. Solve for X
  • Lost controlpanelsettings flag – Enables logging on nodes that are having issues
  • Coinbase Canceling only takes Majority of Fed nodes, but also counts audits in tally – Allows it to require half of the Authority Set to cancel a payment.
  • Create an exception for a factoid transaction timing on block 160181 – Allow nodes downloading the blockchain to progress past block 160180.
  • Cleanup attack/bug vector- Hardens the protocol against error and malicious attack.
  • Remove Replay Data From Save state – Corrects why SaveDBStateList and Restore.save() cause an increase in memory allocation.
  • Misc logging fixes – This improved the quality and level of detail in some log files and add millisecond time resolution to logged times to facilitate debugging.
  • Community Contribution Bug Fix to correct wallet-balances call in factom-walletd – Allows passwords to be used.  Needed by an exchange.
  • Election in minute 9 cause the leader who was voted out to only follow by blocks and never accept the updated authority set. – Allow a Federated Server which was faulted out during minute 9 to recognize that after the blockchain progresses.
  • Force chain head to block 160184 if the head is stuck at 160183 – Compensate for database corruption during multiple restarts.
  • Consider blocks loaded from database as valid – Allows leaders to boot to the same height as followers and audit servers
  • Drop messages older than boot time – Makes network more resilient during restarts
  • GetVirtualServers() could be called in minute 10 and panic – Allows panic in uncommon situations
  • Smarter limits in unmarshalling functions – This is a better fix for a critical bug/attack vector
  • Community Bug Fix: error string out of scope for entryblock panic –  Allows panics to print a message when encountering some types of errors.
  • Reboot issues if dbstate old – This fix is necessary should the network stall for more than an hour.  This allows a network to be rebooted, by allowing the processing of the DBSigs, which set the time frame for a new block.
  • ignore useless message entryblockresponse missingentryblocks – Simplify p2p network interaction by ignoring extraneous discontinued historical messages. These were initially thought to be a severe attack vector against individual nodes, but after fixing were determined to not to be particularly dangerous.
  • factomd panics indicating missing identity entry blocks – Reliably reboot after a partial prior boot

Work Completed

New capabilities

  • Development of 22 number of new capabilities completed and moved into testing for next release listed above

 

Maintenance

  • Development of a number of maintenance corrections completed and moved into testing for next release

 

HR Updates

  • New Hire: Michael Brennan – Development Operations

Future Plans

Protocol Grant

  • Enable the ability to count votes of standing parties
  • Ongoing Performance Improvements
  • Ongoing Maintenance corrections
  • Continue supporting the protocol and issues that arise

Oracle Grant

  • Improvements to Rate Change Log allowing validation against historical data
  • Rate change alerts

Anchor Grant

  • Ethereum Anchors

Awareness


Also published on Medium.

POSTED: January 11, 2019 BY Kevin Casper IN Technical Updates
ABOUT THE AUTHOR

Factom Inc's community manager, carrying a load of marketing operations experience from the enterprise tech and video gaming industries.