Grant Update October 2019

October was a great month for Factom core development. This month the updated Xuan version passed quality control, passed the testnet, and was deployed across the network leaders. This was a huge success because it represented several months of updates and bug fixes going live to make the network more robust.

There were two sponsor meetings in October. One of them was recorded and published: The sponsors in attendance in October were Nolan Bauer, David Kuiper, and Valentin Ganev.

During September there was an attempted release of the Xuan release (v6.4.2). The workarounds that were successful to get around a bug that worked on the testnet ended up not working on the mainnet. During October a patch was developed and the multi-week testing and deployment process started again. By mid-October the v6.4.3 release was ready for mainnet deployment.

The network is becoming more resilient as more development is happening. During the Xuan deployment, a new upgrading strategy was tried. As the software is getting better at handling error conditions, an easier, safer upgrade process has become available. In the past, the safest way to upgrade was by doing a brainswap. This is a manual process that has the potential to cause the network to go into a fail-safe pause. Now that various bugs have been fixed, the Authority Node Operators can take more advantage of the inbuilt faulting mechanisms to assist with the upgrade. A new upgrade procedure was tried, where servers building the blockchain were simply switched off rather than the manual process of upgrading in place. Much thanks goes to Community member Tor who detailed the upgrade process here.

One of the bugs fixed in Xuan was a sequential election error. If multiple Federated servers were to stop working during the same minute, then the software would not handle the recovery well at all. Xuan fixes that bug where after the second election, an Audit server is able to correctly calculate which subset of the blockchain (VM) it is responsible for. Before the fix the Audit server had a high probability of filling in the wrong whole that was made by the second missing Federated server.

Partway during deployment of the 6.4.3 Xuan code, there was one of these multiple sequential elections events. Since the updated software was only partially deployed, the network did not seamlessly recover, but since enough of the network had upgraded, it eventually pulled through the error. This shows how the code is getting more robust against failures as more development is happening.

As a demonstration of how much development is happening, this month’s Github and Jira reports are also available.

Github report:

Jira report:

Another development effort that happened in October was extending the Livefeed API. While much of the implementation of the Livefeed was done by Sphereon, there were still some additional features that needed to make it more useful in a production environment. Factom, Inc added a feature to the Livefeed which allows it to play back older messages that would be missed when computers go offline or are rebooted, etc. This introduces a new way to read data from the blockchain that allows more responsive consumption, and allows event driven applications to be built on the blockchain.

A large development effort during October was digging into the Threading Refactor. The last grant update introduced the design document, and we have started laying the groundwork for the rewrite. One of the more interesting things that was started in October was to simplify boilerplate code management. Golang, the language that factomd is written in lacks the feature of inheritance, which many modern computer languages have. This would have been an easy way to have a small amount of code which is replicated across multiple different files. Golang instead has a feature where code can be automatically generated from templates. This feature was taken advantage of for building the code that implements the inter-process queues. This is represented on the diagram by the lines between boxes.

The next large factomd release is called A4, named by community developer WhoSoup. The process for assembling the various feature branches together into a combined version began in October. The combination process introduced some bugs which were in the process of being debugged at the end of October.


New capabilities and maintenance issues that were released

  • FD-1207
    • Simplify the Timer code
    • Keep federated servers in sync when building blocks.


In Progress

  • FD-1238
    • pokemon bug in CommitEntryMsg
    • Keep factomd running without random crashes
  • FD-1224
    • Return status code 200 for JSONRPC errors in the factomd api
    • we can better conform to RPC standard behavior by setting a 200 response code even during errors
  • FD-1223
    • Allow correct API usage regardless of Host header
    • this removes some unexpected behavior for forwarded ports added in xuan
  • FD-1221
    • Community Contribution – include some factomd changes from Factable Solutions
    • Add clarity around the code base for future developers
  • FD-1217
    • FilterAPI Test Issue
  • FD-1215
    • Update API version to more closely reflect semver
    • Give factomd API users info on what services are offered by a particular version of factomd
  • FD-1210
    • Change priority to to have higher priority than acks.
  • FD-1209
    • Add commit data to ack in wsapi
  • FD-1206
    • Faster Catchup by MMR
    • Faster loading of the first block after boot built by MMR.
  • FD-1204
    • extend Logging to track sizes of Maps and Channels
  • FD-1198
    • MMR catchup delay
  • FD-1195
    • Add logging for every recovery call
    • extending logging to capture this previously silent behavior aids in debugging
  • FD-1194
    • Update build environments to golang 1.13
    • Release software using the latest version of golang
  • FD-1189
    • Add sim test where to-be-elected audit is behind at moment of election
    • Extending sim tests around election/consensus code allows for better regression or checking for un-expected behavior
  • FD-1188
    • Community Contribution – Add documentation around directory and admin blocks
    • Increased understanding when newcomers review source code
  • FD-1175
    • Build factomd docker images with golang 1.13
    • Build images used by ANOs using the latest golang version
  • FD-1166
    • Align grant height to grant payout boundary when reading grant descriptors.
  • FD-1165
    • Update docs for debug API
    • need to update API docs so other users can make use of the new features.
  • FD-1152
    • POC Jenkins builds against DevNet
    • Adding more automation around Integration testing in this POC will help us finish designing the rest of our CI/CD pipeline for Factomd
  • FD-1132
    • Add an RPC api for interacting w/ Nodes on Devnet
    • As a developer I’d like to be able to write test scenarios against DevNet in the same manner as Local Simulation tests.
  • FD-1131
    • DevNet Developer toolbox
    • Having a standard kit for interacting w/ DevNet will make it easier for all developers to make reproducible test scenarios.
  • FD-1123
    • Distribute commits across VMs
  • FD-1122
    • Move Entry Credit balances to own thread
    • Move EC balances to their own thread for future vm threading
  • FD-1101
    • Create framework to execute code on minute edges to consolidate state change code.
    • Simplifying the code to allow better code readability and reliability for EOM processing to prevent race conditions
  • FD-1037
    • Community Contribution – More tests around Audit Server Brainswaps
    • Better debugging problems around brain swapping a follower to an Audit server.
  • FD-1033
    • Log peer behavior. Fix double p2p close.
  • FD-984
    • Create a separate thread to handle all missing data requests.
  • FD-808
    • Updates to enterprise-wallet
    • Include the proposed changes to entreprise-wallet from Factomize
  • FD-677
    • Gossip Improvements
    • Reduce the redundant network traffic caused by Gossip protocol
  • FD-1216
    • ECBlock in Factom lib cannot parse EC balance increases
  • FD-1214
    • Documentation Updates for FD-1213: Walletd sign-data RPC
  • FD-1213
    • Who’s PR for walletd: API call to sign arbitrary data
  • FD-1075
    • factom-cli get dbheight abheight echeight and fbheight are failing to parse
  • FD-1074
    • EBlock is missing in get chainhead on v1-rollup
  • FD-1071
    • Missing Sequence Number in get head call of v1-rollup-testing
  • FD-1062
    • factom-walletd build fails with webServer.Config.CorsDomains undefined
  • FD-1059
    • Adding circle CI to factom lib
    • Adding an automated building + test suite to the factom library will improve the stability of the library.
  • FD-1050
    • Help shows -S option available for factom-cli diagnostic network command but the same doesn’t work
  • FD-1044
    • The results of auto-complete and regular help are different
  • FD-1042
    • factom lib; add version field to Entry type
  • FD-1031
    • factom-cli command completion
  • FD-873
    • add factom-cli command for diagnostics
  • FD-872
    • add factom-cli command for authorities
  • FD-881
    • add factom-cli command for get tps
  • FD-1005
    • add factom-cli display options for get head (dblock)
  • FD-855
    • add factom-cli command for get ecblock
  • FD-1002
    • add factom-cli display options for get entry, get firstentry
  • FD-877
    • add factom-cli command to get fblock
  • FD-871
    • add factom-cli command for get ablock
  • FD-999
    • add factom-cli display options for get dblock
  • FD-997
    • factom lib merge identity key functions from master into v1-rollup
  • FD-855
    • add factom-cli command to get current-minute information
  • FD-963
    • Improve code documentation in factom lib
  • FD-961
    • organize the factom lib codebase
    • Makes working with factom lib easier for developing factom client applications
  • FD-904
    • NewEntry functions
  • FD-920
    • replace the “byHeight” functions in factom lib
    • makes factom lib much more useful when querying blocks from factomd
  • FD-900
    • Missing an easy function to convert the fields to a chainID
  • FD-899
    • Community Contribution – add -config option to set factom-walletd config file path
    • Allow factom-walletd to use a config file loaded from a configurable path.
  • FD-881
    • Add missing factom lib caller for tps-rate
    • improves functionality for creating factom clients
  • FD-880
    • Add missing factom lib caller for multiple-fct-balances
    • improves functionality for creating factom clients
  • FD-879
    • Add missing factom lib caller for multiple-ec-balances
    • improves functionality for creating factom clients
  • FD-877
    • Add missing factom lib caller for factoid-block
    • improves functionality for creating factom clients
  • FD-875
    • Add missing factom lib caller for entrycredit-block
    • improves functionality for creating factom clients
  • FD-873
    • Add missing factom lib caller for diagnostics
    • improves functionality for creating factom clients
  • FD-872
    • Add missing factom lib caller for authorities
    • improves functionality for creating factom clients
  • FD-871
    • Add missing factom lib caller for admin-block
    • improves functionality for creating factom clients
  • FD-855
    • Add missing factom lib caller for current-minute
    • Ensure API’s are updated on Repo for supportability
  • FD-648
    • wallet database reinitializes
    • automatically clear the cache of factom-walletd when detected a new local test-blockchain.
  • FD-335
    • let factom-walletd discover if it is on a new blockchain
    • this is related to other wallet issues selected to increase wallet stability and functionality

Future Plans

Protocol Grant

  • Ongoing Performance Improvements

  • Ongoing Maintenance corrections

  • Continue supporting the protocol and issues that arise

Oracle Grant

  • Ongoing Maintenance corrections

Anchor Grant

  • Ongoing Maintenance corrections