08/11/2021, 12:10 08/11/2021, 12:10 IN NEAR The NEAR White Paper – NEAR Protocol The NEAR White Paper — NEAR Protocol The NEAR White Paper The NEAR White Paper SECTION 01 SECTION 01 Abstract Abstract NEAR is a decentralized application platform with the potential tochange how systems are designed, how applications are built and how the web itself works. NEAR is a decentralized application platform with the potential to change how systems are designed, how applications are built and how the web itself works. It is a complex technology with a simple goal — allow developers and entrepreneurs to easily and sustainably build applications which secure high value assets like money and identity while making them performant and usable enough for consumers to access. It is a complex technology with a simple goal — allow developers and entrepreneurs to easily and sustainably build applications which secure high value assets like money and identity while making them performant and usable enough for consumers to access. To do this, NEAR is built from the ground up to deliver intuitive experiences for end users, scale capacity across millions of devices and provide developers with new and sustainable business models for their applications. In doing so, NEAR is creating the only community-run cloud strong enough to extend the reach of Open Finance and power the future of the Open Web. The following sections will describe the approach NEAR takes to designing and implementing the core technology of its system. Wherever possible, we will use language that is accessible and we will describe relevant sections starting from first principles, values and design intent before digging into their technical implementation. Additional depth on technical topics can be found in the relevant topic- specific papers and blog posts. It must be noted that, as with all complex systems under active development, the contents of this guide and the technology they explain are both subject to change. In fact, one of the hallmarks of the NEAR approach is rapid and pragmatic iteration. The latest information about the protocol can be found in posts on the blog at https://www.near.org/blog, live chat channels at http://near.chat and in the reference code base at https://github.com/nearprotocol. https://near.org/papers/the-official-near-white-paper/ htencinoarorg/canore/the offcicl-near-white-paper/ 1/45 1/45 To do this, NEAR is built from the ground up to deliver intuitive experiences for end users, scale capacity across millions of devices and provide developers with new and sustainable business models fortheir applications. In doing so, NEAR is creating the only community-run cloud strong enough to extend the reach of Open Finance and power the future of the Open Web. The following sections will describe the approach NEAR takes to designing and implementing the core technology of its system. Wherever possible, we will use language that is accessible and we will describe relevant sections starting from first principles, values and design intent before digging into *hair technical implementation. Additional depth on technical topics can be found in the relevant topic- specific papers and blog posts. It must be noted that, as with all complex systems under active development, the contents of this guiue and the technology they explain are both subject to change. In fact, one ofthe hallmarks ofthe NEAR approach is rapid and pragmatic iteration. The latest information about the protocol can be found im pests cn tne viog at https://www.near.org/blog, live chat channels at http://near.chat and in the roterence coac base at https://github.com/nearprotocol. 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol SECTION 02 Introduction The richness of today’s web emerged from the combined efforts of millions of people taking advantage of “permissionless innovation” — the ability to create content and applications without asking anyone first. Unfortunately, the lack of freedom for data has resulted in an environment which is actively adversarial to the interests of its participants. A small number of companies have enticed vast numbers of users to join by luring them in with network effects and then captured them by holding their data to prevent them from seeking alternatives. Similarly, these massive platforms have enticed applications to build atop their ecosystems before either cutting off access or actively opposing their interests when the applications became too successful. As a result, these walled gardens have stifled innovation and effectively monopolized vast sections of the web. In the future, we can fix this by using new technologies to re-enable the permissionless innovation of the past in a way which creates a more open web where users are free and applications are supportive rather than adversarial to their interests. We have already seen the power of this kind of freedom in the financial sector, where decentralized digital currencies like Bitcoin and their underlying blockchain technology have facilitated billions of dollars of peer-to-peer transfers at a fraction of the price of the traditional banking system. The same underlying technology also allows participants in the $50B+ virtual goods economy to track, take ownership and trade these goods permissionlessly among themselves. It allows real world goods to cross into the digital realm, with verified ownership and tracking just like the digital ones. Beyond what we’ve seen today, a web where freedom of data enables permissionless innovation by default will drive a new form of software development. In this web, developers can quickly construct applications from open state components and power their efforts with new business models which are enabled from within the software itself rather than rely on parasitic relationships with their users. This doesn’t just accelerate the creation of applications which have a more honest and collaborative relationship with their users but it also allows for the emergence of entirely new businesses built on top of them. These new applications and the open web that powers them can only be enabled by the right kind of infrastructure. The platform of the new web cannot be controlled by a single entity nor have its usage limited by insufficient scalability. It must be as decentralized in design as the web itself and supported by a widely distributed community of operators so the value it stores cannot be censored, modified or removed without the permission of the users on whose behalf that value is stored. It should be secure https://near.org/papers/the-official-near-white-paper/ 2/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol and stable enough to form the backbone of the new economy. This is the infrastructure of NEAR. NEAR is a decentralized application platform which is designed to enable the open web of the future and power its economy. It uses the same core underlying technology that made Bitcoin an unkillable currency and combines it with cutting edge advances in community consensus, database sharding and usability. On this web, everything from new currencies to new applications to new industries can be created, opening the door to a brand new future. Why decentralization matters On the surface, many of the design goals of a decentralized blockchain-based platform can be accomplished both faster and cheaper by using existing platforms. For example, the cost to store data or perform computation on the Ethereum blockchain are between thousands and millions of times higher than the cost of performing the same functions on Amazon’s Web Services. A developer can always build a “centralized” application or even a centralized currency at a fraction of the cost of doing the same on a decentralized platform because the decentralized platform will, by definition, have many redundancies in its processes and storage. Why is it important to pay the added cost to support decentralization? Because not all data is created equal. Certain elements of value, for example the bits representing ownership of digital currency, personal identity or titles to assets, are highly sensitive. In a centralized system, the following players can all directly change the value of any balances they come into contact with: 1. The developer who controls the release or update of the application’s code 2. The platform where the data is stored 3. The servers which run the application’s code Even if none of these players intend to operate with bad faith, the actions of governments, police forces and hackers can easily turn their hands against their users and censor, modify or steal the balances they are supposed to protect. A typical user will trust a typical centralized application, despite its potential vulnerabilities, with everyday data and computation. Typically, only banks and governments are trusted sufficiently to maintain custody of the most sensitive information — balances of wealth and identity. But these entities are also subject to the very human forces of hubris, corruption and theft. For example, the Global Financial Crisis in 2008 showed the fundamental problems of trusting an over- leveraged banking system. It also provided a timely example of how governments around the world implement substantial capital controls on citizens during times of crisis. Beyond this example, it has https://near.org/papers/the-official-near-white-paper/ 3/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol become a truism that hackers now likely own most or all of your sensitive data. By contrast, a fully decentralized system doesn’t have an “off” switch and it doesn’t have a way for nefarious forces to impose their will on the applications built on top of it. To accomplish this, the system requires substantial redundancy in both computation and storage of data because any points of failure in these areas can be exploited. The more sensitive the information being stored, the more redundancy and security is required… and the more decentralization matters. Blockchain-based systems are the substrate for this decentralization because their immutability provides the primitives — tokens, for example — necessary to incentivize cooperation and coordination among the numerous actors who make up these systems and power their redundancy. Once these systems are launched, they become essentially “unkillable”. The benefits of building applications on top of such a system are substantial. Not only is highly sensitive information secured and available globally, but currency is now a native primitive of the medium. These decentralized applications operate on a more complex infrastructure than today’s web but they have access to an instantaneous and global pool of currency, value and information that today’s web, where data is stored in the silos of individual corporations, cannot provide. As importantly, the data these apps secure is fully owned and controlled by their end users rather than the apps themselves. This opens up a wealth of new use cases which could not exist without a decentralized infrastructure. While decentralization is crucially important, not all blockchain-based systems are decentralized. Decentralization is a scale which can be measured along a number of dimensions but, fundamentally, it comes down to how many players in the system must be corrupted in order to break the system itself and how likely that is to occur. The more important the assets the system must protect, the more important it is that true decentralization is achieved rather than a system which merely pays it lip service. Later sections will describe the technical architecture which achieves decentralization for NEAR. A brief summary of NEAR NEAR is a decentralized application platform which runs atop the NEAR Protocol blockchain. This blockchain, which runs across hundreds of machines around the world, is organized to be permissionless, performant and secure enough to create a strong and decentralized data layer for the new web. Essentially, NEAR is a platform for running applications which have access to a shared — and secure — pool of money, identity and data which is owned by their users. More technically, it combines the features of partition-resistant networking, serverless compute and distributed storage into a new kind of platform. For comparison, Amazon’s Web Services and Microsoft’s Azure operate much of the infrastructure of the web today and are two of the most common “clouds” where applications are deployed. Each of the individual servers which make up these computing and storage clouds are controlled by a single entity. This means that anything run on or stored within them is completely at the mercy of those companies https://near.org/papers/the-official-near-white-paper/ 4/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol or the government agencies which require them to do things against their will. Data can easily be lost, censored, altered, sold or hacked. This is because there is only a single point of failure. Applications which are deployed to these cloud servers can also be continuously modified by their original developers or whoever holds their credentials. This makes software updates easy for developers but it also means that any data accessible by an application can be censored, modified or stolen by these same developers, whether at their own direction or because they were hacked or forced by a governmental authority to do so. Because user data is generally stored in large pooled databases, these developers become juicy targets for exactly those activities. Together, the vulnerability of the developers and the platforms themselves makes any sensitive data stored on these platforms vulnerable. When the cloud which hosts these applications is instead run by a global community which anyone can be a part of, the programs and assets stored by it become transparent and essentially “unkillable”, allowing users to store meaningful things like money, identity and digital assets and securely transact them with anyone without requiring someone else’s permission or platform. There is no single point of failure because there are multiple redundancies around the world and there is security because of the consensus which is programmatically achieved among community members who make up the cloud network. To eliminate the vulnerability of the developer, applications deployed to this cloud can be programmatically locked so no further updates can modify the state they access. Essentially, once they achieve this state, they become autonomous and can be trusted to continue to perform their functions without fail or interference. This allows the secure storage of high value assets like money, identity and key pieces of data. Bitcoin can be thought of as the first, very basic, version of this global community-run cloud, though it is primarily used only to store and move the Bitcoin digital currency. Ethereum is the second and slightly more sophisticated version, which expanded the basic principles of Bitcoin to create a more general computing and storage platform, though it is a raw technology which hasn’t achieved meaningful mainstream adoption. NEAR represents an evolution beyond what has come before it and is the first decentralized application platform to solve all of the three key challenges to gaining mainstream adoption: usability, scalability, and security. Challenges of Creating a Community Cloud A community-run system like this has very different challenges from centralized “cloud” infrastructure which is run by a single entity or group of known entities. For example: https://near.org/papers/the-official-near-white-paper/ 1. It must be both inclusive to anyone and secure from manipulation or capture. 2. Participants must be fairly compensated for their work while avoiding creating incentives for negligent or malicious behavior. 3It tbbth th ti ll d t fidth i ht ilibi d i t t 5/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol 3. It must be both game theoretically secure so good actors find the right equilibrium and resistant to manipulation so bad actors are actively prevented from negatively affecting the system. A free-market-based system is the only way to seamlessly align these incentives and so the NEAR Platform uses a token — also called “NEAR” — to glue it all together. This token allows the users of these cloud resources, regardless of where they are in the world, to fairly compensate the providers of the services and to ensure that these participants operate in good faith. To remain decentralized, it’s important that a community-run system like this be permissionless, meaning anyone has the opportunity to participate. To ensure this, anonymity is crucial and so revealing a party’s identity is not required. While this provides for decentralization, it also opens up a wide range of misbehavior so all the mechanisms of the system must assume that one individual actor might be controlling a single account or a million accounts. Thus we operate with a principle of “one token equals one vote” to participate and govern the system. These systems must also balance the need to be robustly decentralized with the reality that technologies and communities must be given the freedom to iterate or risk being rendered obsolete. Thus the long term health of the community requires maintaining a broad degree of decentralization and strong security guarantees in the platform itself while also creating efficient processes for evolving its technology over time. SECTION 03 Why NEAR? Today’s blockchains have achieved significant progress — Bitcoin, the original blockchain which launched in 2008, is a store of value whose network has been priced at over $300 billion while Ethereum, the original “global computer” which launched in 2014, boasts thousands of innovative applications spanning from gaming to decentralized finance. Unfortunately, neither these original networks nor any of those which followed have managed to bridge the gap towards mainstream adoption of the applications which are built on top of them nor provide the kind of scale which supports an entire Web. This is a result of two key factors: 1. System design 2. Organization design System design is relevant because the technical architecture of other platforms creates substantial problems with both usability and scalability which have made adoption nearly impossible by any but the https://near.org/papers/the-official-near-white-paper/ 6/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol most technical innovators. End-users experience 97-99% dropoff rates when using applications and developers find the process of creating and maintaining their applications endlessly frustrating. Fixing these problems requires substantial and complex changes to current protocol architectures, something which existing organizations haven’t proven capable of implementing. Instead, they create multi-year backlogs of specification design and implementation which result in their technology falling further and further behind. NEAR’s platform and organization are architected specifically to solve the above mentioned problems. The technical design is fanatically focused on creating the world’s most usable and scalable decentralized platform so global-scale applications can achieve real adoption. The organization and governance structure are designed to rapidly ship and continuously evolve the protocol so it will never become obsolete. The following section will highlight the key features which address these problems. Key Features Each of the key problems faced by current platforms, their developers and their end-users are addressed below. More information about the specific implementation of these features is left to the following sections of this paper. Usability First New blockchain-based platforms generally claim to differentiate themselves based on providing scalability relative to existing platforms. The scalability of a platform isn’t relevant, however, unless the platform has sufficient adoption to require that throughput. As an analogy, it makes no sense to create an enormous stadium that can seat 100,000 people in the middle of the desert where no one wants to go in the first place. Thus, the more important immediate problem to address is how to allow developers to easily create useful applications that users can actually use and which will capture sustainable value for those developers. While some changes along these dimensions can be handled in the second layer of the technical stack, the most important ones must be made at the protocol level and cannot be bolted on afterward. End-User Usability Developers will only build applications which their end users can actually use. NEAR’s “progressive security” model allows developers to create experiences for their users which more closely resemble familiar web experiences by delaying onboarding, removing the need for user to learn “blockchain” concepts and limiting the number of permission-asking interactions the user must have to use the application. 1. Simple Onboarding: NEAR allows developers to take actions on behalf of their users, which allows https://near.org/papers/the-official-near-white-paper/ 7/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol them to onboard users without requiring these users to provide a wallet or interact with tokens immediately upon reaching an application. Because accounts, which have human-readable names like foobar.near, keep track of application-specific keys, user accounts can also be used for the kind of “Single Sign On” (SSO) functionality that users are familiar with from the traditional web (eg “Login with Facebook/Google/Github/etc”). 2. Easy Subscriptions: Contract-based accounts allow for easy creation of subscriptions and custom permissioning for particular applications. 3. Familiar Usage Styles: The NEAR economic model allows developers to pay for usage on behalf of their users in order to hide the costs of infrastructure in a way that is in line with familiar web usage paradigms. 4. Predictable Pricing: NEAR prices transactions on the platform in simple terms which allow end- users to experience predictable pricing and less cognitive load when using the platform. Developer Usability A number of key usability improvements support developers of the platform, allowing them to more easily learn, develop, test and deploy their applications than they can with any other platform. 1. Familiar Languages: NEAR nodes run Web Assembly (WASM), which can be compiled from a host of popular languages. Initially, smart contracts can be built using Rust, a very secure and comprehensive programming language that is rapidly gaining popularity. NEAR also supports contracts written in AssemblyScript which is very similar to TypeScript, a Microsoft-developed modification of JavaScript that has types and a very broad adoption among developers. In the future, more common programming languages will be supported so developers don’t have to learn an entirely new language to build applications atop the platform. 2. Robust Tooling: NEAR’s development suite is created to support the developer workflow with a unified set of tools so developers can easily build, test and deploy applications. The tooling on top of the platform and the APIs exposed by it provide developers with the kind of development experience they are used to from traditional web apps. This includes one-click deploy, integrated unit testing, easy front-end integration and debugging from the web browser’s developer console. 3. Developer Business Models: The NEAR Protocol supports developers by helping them monetize the open components they create for the ecosystem by natively rewarding them with rebates based on the usage of those components. This is addressed specifically in a following section. 4. Predictable Pricing: NEAR prices transactions on the platform in simple terms which allow the developer to experience predictable pricing and less cognitive load when using the platform. Scalability Second A future-proof protocol must shard both state and processing in order to scale. With significant adoption of the platform, no single machine would otherwise be capable of storing all the information on the chain or verifying all of the transactions. NEAR uses a sharding approach which allows the network to increase its capacity as additional nodes participate. This is done by dynamically splitting the network nodes into multiple shards when usage is https://near.org/papers/the-official-near-white-paper/ 8/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol high enough to require it and parallelizing computation across those shards. With this approach, the network can scale continuously as demand increases. A lot of recent sharding research in the blockchain community separates transactions into intra-shard and cross-shard categories, optimizing for the former and providing a much slower solution for the latter. The NEAR Protocol assumes that transactions will touch multiple shards by default, which is the likely behavior for arbitrary smart contracts, and optimizes performance accordingly. Efficient Development and Evolution A key challenge facing both existing and new platforms is how they handle development and evolution of the platform. While the platform itself must be decentralized, there are multiple approaches to updating and evolving it. Existing protocols have proven insufficiently iterative to keep up with the pace of innovation and a successful next-generation protocol must ensure that the network maintains its decentralization while still allowing for an efficient development process which prevents it from being disrupted by the next wave of technology. NEAR’s initial development is being done by one of the strongest teams of engineers, entrepreneurs and technologists in the world and its governance is designed to ensure that the protocol is developed subject to ongoing community oversight but with sufficient efficiencies in the process that it will remain competitive and relevant long after its launch. This ensures that NEAR will not only avoid the “failure to launch” problem which has plagued one half of the industry but also the “failure to improve” problem which has held back the other half. Real Decentralization Even though Bitcoin and Ethereum are often lauded for their level of decentralization, they actually suffer from many centralization issues. Bitcoin, for example, has 53% of its mining power controlled by just three pools. On top of that, running mining nodes currently requires expensive hardware, which increases barriers to entry and reduces the incentives for nodes to join over time. Newer networks often trade the hope of decentralization for the operational efficiencies provided by more centralized implementations which use either limited validator sets or fully “permissioned networks”. This violates one of the fundamental tenets of a truly decentralized network — that its value is protected by its level of redundancy among independent nodes. In order to maintain real decentralization, the network needs to allow permissionless participation from prospective node operators and not incentivize pooling. To address these concerns, NEAR uses a staking mechanism called “Thresholded Proof of Stake” which is specifically designed to be both deterministic and broadly fair so it doesn’t incentivize pooling of large validators and it encourages broad participation from nodes. Lowering the barriers to entry for nodes accomplishes more than simply decentralizing the network. In ahorizontallyscalingsystemlikeNEAR’sthemorenodeswhichcanparticipatethemoreitcanscale https://near.org/papers/the-official-near-white-paper/ 9/45 08/11/2021, 12:10 a horizontally scaling system like NEARs, the more nodes which can participate, the more it can scale as well. The NEAR White Paper – NEAR Protocol A New Business Model for Developers and Entrepreneurs An early use case for blockchains like Ethereum was allowing projects to create their own tokens and raise funds through “Initial Coin Offerings” (ICOs). This initially appeared to be a revolutionary new way of allowing infrastructure developers to access capital and to bootstrap network effects for their projects, something which had long been lacking in the world of open source software and infrastructure. Unfortunately, creating application-layer tokens also put major usability hurdles in front of users and the willful speculation and fraud that subsequently occurred made it clear that this was not a viable path forward for most developers. NEAR provides developers and entrepreneurs with a more robust, less intrusive and more legitimate way of monetizing their infrastructure. When a contract is called, a portion of the fees generated by the network are automatically allocated to that contract and might (if coded as such) be withdrawn by its developer. This both incentivizes early infrastructure development (because the early contracts will build network effects that increase usage) and provides a business model so application and infrastructure developers can benefit from their creations without creating ill-advised tokens of their own. SECTION 04 Design Principles Both the design and development of the NEAR platform are guided by a handful of key principles. These principles reflect the problems inherent in both the centralized and decentralized systems of today. 1. Usability: Applications deployed to the platform should be seamless to use for end users and seamless to create for developers. Wherever possible, the underlying technology itself should fade to the background or be hidden completely from end users. Wherever possible, developers should use familiar languages and patterns during the development process. Basic applications should be intuitive and simple to create while more robust applications should still be secure. 2. Scalability: The platform should scale with no upper limit as long as there is economic justification for doing so in order to support enterprise-grade, globally-used applications. 3. Simplicity: The design of each of the system’s components should be as simple as possible in order to achieve their primary purpose. Optimize for simplicity, pragmatism and ease of understanding above theoretical perfection. https://near.org/papers/the-official-near-white-paper/ 4. Sustainable Decentralization: The platform should encourage significant decentralization both i th h tt dthl t i d t l th l ith t Th l tf 10/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol in the short term and the long term in order to properly secure the value it hosts. The platform — and community — should be widely and permissionlessly inclusive and actively encourage decentralization and participation. To maintain sustainability, both technological and community governance mechanisms should allow for practical iteration while avoiding capture by any single parties in the long run. SECTION 05 How NEAR Works NEAR provides a community-operated cloud infrastructure for deploying and running decentralized applications. It combines the features of a decentralized database with others of a serverless compute platform. The token which allows this platform to run also enables applications built on top of it to interact with each other in new ways. Together, these features allow developers to create censorship resistant back-ends for applications that deal with high stakes data like money, identity and assets and open-state components which interact seamlessly with each other. These application back-ends and components are called “smart contracts,” though we will often refer to these all as simply “applications” here. The infrastructure which makes up this cloud is created from a potentially infinite number of “nodes” run by individuals and organizations around the world who offer portions of their CPU and hard drive space — whether on their laptops or, more likely, professionally deployed servers. Developers write smart contracts and deploy them to this cloud as if they were deploying to a single server, which is a process that feels very similar to how applications are deployed to existing centralized clouds. Once the developer has deployed an application, called a “smart contract”, and marked it unchangeable (“immutable”), the application will now run for as long as at least a handful of members of the NEAR community continue to exist. When end users interact with that deployed application, they will generally do so through a familiar web or mobile interface just like any one of a million apps today. In a centralized cloud hosted by Amazon or Google, developers pay for their applications each month based on how much usage they required, for example based on the number of requests generated by users visiting their webpages. The NEAR platform similarly requires that either users or developers provide compensation for their usage to the community operators of this infrastructure. Like today’s cloud infrastructure, NEAR prices usage based on easy to understand metrics that aren’t heavily influenced by factors like system congestion. Such factors make it very complicated for developers on alternative blockchain-based systems today. More details on the economics of NEAR can be found in the Economics section. https://near.org/papers/the-official-near-white-paper/ 11/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol In a centralized cloud, decisions are made unilaterally by the controlling corporation. The NEAR community-run cloud is decentralized so updates must ultimately be accepted by a sufficient quorum of the network participants. Updates about its future are generated from the community and subject to an inclusive governance process which balances efficiency and security. More details of the governance process can be found in the Governance section. In order to ensure that the operators of nodes — who are anonymous and potentially even malicious — run the code with good behavior, they participate in a staking process called “Proof of Stake”. In this process, they willingly put a portion of value at risk as a sort of deposit which they will forfeit if it is proven that they have operated improperly. More details of the staking process can be found in the Technology section. Elements of the NEAR Platform The NEAR platform is made up of many separate elements. Some of these are native to the platform itself while others are used in conjunction with or on top of it. The NEAR Token NEAR token is the fundamental native asset of the NEAR ecosystem and its functionality is enabled for all accounts. Each token is a unique digital asset similar to Ether which can be used to: 1. Pay the system for processing transactions and storing data. 2. Run a validating node as part of the network by participating in the staking process. 3. Help determine how network resources are allocated and where its future technical direction will go by participating in governance processes. The NEAR token enables the economic coordination of all participants who operate the network plus it enables new behaviors among the applications which are built on top of that network. Other Digital Assets The platform is designed to easily store unique digital assets which may include, but aren’t limited to: Other Tokens: Tokens bridged from other chains (“wrapped”) or created atop the NEAR Platform can be easily stored and moved using the underlying platform. This allows many kinds of tokens to be used atop the platform to pay for goods and services. “Stablecoins,” specific kinds of token which are designed to match the price of another asset (like the US Dollar), are particularly useful for transacting on the network in this way. Unique Digital Assets: Similar to tokens, digital assets (sometimes called “Non Fungible Tokens” https://near.org/papers/the-official-near-white-paper/ 12/45 08/11/2021, 12:10 q g , g The NEAR White Paper – NEAR Protocol ( g (NFTs) ranging from in-game collectibles to representations of real-world asset ownership can be stored and moved using the platform. The NEAR Platform The core platform, which is made up of the cloud of community-operated nodes, is the most basic piece of infrastructure provided. Developers can permissionlessly deploy smart contracts to this cloud and users can permissionlessly use the applications they power. Applications, which could range from consumer-facing games to digital currencies, can store their state (data) securely on the platform. This is conceptually similar to the Ethereum platform. Operations atop the platform which require computation, network usage or storage require payment to the platform in the form of fees which the platform then distributes among its community of validating nodes. These operations can include the creation of new accounts, the deployment of new contracts, the execution of code by a contract and the storage or modification of data by a contract. Details of these costs is laid out in the Economics section. Details of how the nodes work are provided in the Technology section. The platform can be interfaced with permissionlessly. As long as the rules of the protocol are followed, any independent developer can write software which interfaces with it (for example, by submitting transactions, creating accounts or even running a new node client) without asking for anyone’s permission first. The NEAR Development Suite The NEAR platform is designed to be used independently and permissionlessly but a set of tools and reference implementations is being created to facilitate its use by those developers and end users who prefer them. These tools include: NEAR SDKs: NEAR supports Rust and AssemblyScript (JavaScript with types) languages to write smart contracts. To provide a great experience for developers, NEAR has a full SDK which includes standard data structures, examples and testing tools for these two languages. Gitpod for NEAR: NEAR uses existing technology Gitpod to create zero time onboarding experience for developers. Gitpod provides an online “Integrated Development Environment” (IDE), which NEAR customized to allow developers to easily write, test and deploy smart contracts from a web browser. The NEAR Examples website contains templates that can be deployed in one-click to make the process of building on NEAR for both new and old developers as simple as possible. NEAR Wallet: A wallet is a basic place for developers and end users to store the assets they need to use the network. NEAR Wallet is a reference implementation that is intended to work seamlessly with the progressive security model that lets application developers design more effective user experiences. It will eventually include built-in functionality to easily enable participation by holders in staking and governance processes on the network. https://near.org/papers/the-official-near-white-paper/ 13/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol NEAR Explorer: To aid with both debugging of contracts and the understanding of network performance, Explorer presents information from the blockchain in an easily digestible web-based format. NEAR Command Line Tools: The NEAR team provides a set of straightforward command line tools to allow developers to easily create, test and deploy applications from their local environments. All of these tools are being created by the community in an open-source manner so they can be modified or deployed by anyone. SECTION 06 Economics The ecosystem which makes up the NEAR platform is driven primarily by economic forces. This economy creates the incentives which allow participants to permissionlessly organize to drive the platform’s key functions while creating strong disincentives for undesirable, irresponsible or malicious behavior. In order for the platform to be effective, these incentives need to exist both in the short term and the long term. Fundamentally, the NEAR platform is a marketplace between willing participants. On the supply side, operators of the validator nodes and other fundamental infrastructure need to be incentivized to provide these services which make up the “community cloud.” On the demand side, the developers and end-users of the platform who are paying for its use need to be able to do so in a way which is simple, clear and consistent so it helps them. Further, economic forces can also be applied to support the ecosystem as a whole. They can be used at a micro level to create new business models by directly compensating the developers who create its most useful applications. They can also be used at a macro level by coordinating the efforts of a broader set of ecosystem participants who participate in everything from education to governance. The specific application of each of these forces is described in the sections below. We will begin by benchmarking how each of the key design principles of NEAR apply to its economics and survey the landscape of existing approaches. NEAR Economy Design Principles https://near.org/papers/the-official-near-white-paper/ NEAR’s overall system design principles are used to inform its economic design according to the f ll i i t tti 14/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol following interpretations: 1. Usability: End users and developers should have predictable and consistent pricing for their usage of the network. Users should never lose data forever. 2. Scalability: The platform should scale at economically justified thresholds. 3. Simplicity: The design of each of the system’s components should be as simple as possible in order to achieve their primary purpose. 4. Sustainable Decentralization: The barrier for participation in the platform as a validating node should be set as low as possible in order to bring a wide range of participants. Over time, their participation should not drive wealth and control into the hands of a small number. Individual transactions made far in the future must be at least as secure as those made today in order to safeguard the value they modify. Overview The NEAR economy is optimized to provide developers and end users with the easiest possible experience while still providing proper incentives for network security and ecosystem development. Here is a summary of the key ideas that drive the system: 1. Thresholded Proof of Stake: Validating node operators provide scarce and valuable compute resources to the network. In order to ensure that the computations they run are correct, they are required to “stake” NEAR tokens which guarantee their results. If these results are found to be inaccurate, the staker loses their tokens. This is a fundamental mechanism for securing the network. The threshold for participating in the system is set algorithmically at the lowest level possible to allow for the broadest possible participation of validating nodes in a given “epoch” period (½ of a day). 2. Epoch Rewards: Node operators are paid for their service a fixed percentage of total supply as a “security” fee of roughly 4.5% annualized. This rate targets sufficient participation levels among stakers in order to secure the network while balancing with other usage of NEAR token in the ecosystem. 3. Protocol treasury: In addition to validators, the protocol treasury receives 0.5% of total supply annually to continuously re-invest into ecosystem development. 4. Transaction Costs: Usage of the network consumes two separate kinds of resources — instantaneous and long term. Instantaneous costs are generated by every transaction because each transaction requires the usage of both the network itself and some of its computation resources. These are priced together as a mostly-predictable cost per transaction, which is paid in NEAR tokens. 5. Storage Costs: Storage is a long term cost because storing data represents an ongoing burden to the nodes of the network. Storage costs are covered by maintaining minimum balance of NEAR tokens on the account or contract. This provides an indirect mechanism of payment via inflation to validators for maintaining contract and account state on their nodes. https://near.org/papers/the-official-near-white-paper/ 6. Inflation: Inflation is the combination of payouts to validators and the protocol treasury minus the collectedtransactionfees(andafewotherNEARburningmechanicslikethenameauction 15/45 08/11/2021, 12:10 collected transaction fees (and a few other NEAR burning mechanics like the name auction. Overall, the maximum inflation is 5%, which can go down over time as network gets more usage The NEAR White Paper – NEAR Protocol and more transactions fees are burned. It’s possible that inflation becomes negative (total supply decreases) if there are enough fees burned. 7. Scaling Thresholds: In a network which scales its capacity relative to the amount of usage it receives, the thresholds which drive the network to bring on additional capacity are economic in nature. 8. Security Thresholds: Some thresholds which provide for good behavior among participants are set using economic incentives. For example, “Fishermen” (described separately). The justifications for each of these principles is described in more detail in the following sections. Resources Provided A blockchain-based cloud provides several specific resources to the applications which run atop it: Compute (CPU): This is the actual computer processing (and immediately available RAM) which run the code in a contract. Bandwidth (“Network”): This is the network traffic between participants and users, including messages which submit transactions and those which propagate blocks. Storage: Permanent data storage on the chain, typically expressed as a function of storage space (eg kilobytes). Existing blockchains like Ethereum account for all of these in a single up front transaction fee which represents a separate accounting for each of them but ultimately charges developers or users for them only once in a single fee. This is a high volatility fee commonly denominated in “gas”. Developers prefer predictable pricing so they can budget and provide prices for their end users. The pricing for the above-mentioned resources on NEAR is an amount which is slowly adjusted based on system usage (and subject to the smoothing effect of resharding when usage grew sustainably) rather than being fully auction-based. This means that a developer can more predictably know that the cost of running transactions or maintaining their storage. Initially, all of these resources will be priced and paid in terms of NEAR tokens. In the future, they may also be priced in terms of a stable currency denomination (for example a token pegged to the $USD). Compute and Bandwidth (“gas”) Compute (CPU) is a momentary resource spent on executing a transaction. The cost of each CPU instruction is denominated in “gas” units and its price is determined based on the slowly adjusted price of gas (denominated in NEAR tokens). Bandwidth is usually measured in bytes, but in the NEAR platform it is converted into gas units by using a simple coefficient of overhead which has been estimated on reference hardware. https://near.org/papers/the-official-near-white-paper/ 16/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol Each chunk (piece of a block) is given a specific maximum gas limit which is determined based on how much a single block can “fit” when executed on reference hardware. The block limit can also be adjusted by network participants in order to account for performance improvements or the participants deciding to run better hardware. This is done via the normal system governance processes. The current gas price is predictable but not fixed. Each block, it is adjusted in the following way: If the prior block is more than half full, the gas price from the previous block is increased by an amount given by the parameter called `alpha`. If the prior block is less than half full, the gas price from the previous block is decreased by an amount also given by the parameter called `alpha`. Gas usage can trigger Resharding, as explained below. Storage Storage is a long-term scarce asset. For an application or users to use it, they must maintain a minimum balance on their account that scales linearly with amount of storage such account takes. The required amount of NEAR tokens per byte is fixed and is subject to change only by major governance decision (and, given trends in storage hardware and system capacity, it will likely be adjusted down going forward). For example, if some contract takes 10 kilobytes of storage due to data stored under it, this contract must maintain minimum balance of 1 NEAR. A contract like USDT from Ethereum would need to maintain ~10k NEAR balance to cover for storage it is using. This also means that regular user’s accounts need to maintain fraction of NEAR minimum balance. Using a minimum balance of NEAR on the account leads to this amount not being staked or used in other applications. Validators are getting paid indirectly for maintaining this storage from inflation and the fact that total stake is smaller. The NEAR system will keep shard size mostly balanced, allowing for each node to maintain roughly the same amount of state (which will be roughly the total state size divided by the number of shards). As individual shard state size changes, the assignment of accounts and contracts to shards can shift to maintain this balance. Resharding Accounts and contracts are each assigned to a shard. Because the usage of such contracts is not equal, the usage or size of some shards might greatly outpace the usage or size of other shards. To prevent this, NEAR uses resharding, which rebalances shards periodically based on specific conditions. The end result will be a set of shards that are expected to have a reasonable and balanced amount of transactions and storage usage. During each epoch (½ day), statistics are accumulated regarding how full blocks are during that epoch https://near.org/papers/the-official-near-white-paper/ 17/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol and other relevant conditions. Each contract is examined based on its usage during the previous epoch and its current storage. Contracts are then “bucketed” into groups such that each bucket has roughly the same expected aggregate characteristics. The system knows approximate limits on transaction usage per shard (the “gas limit”) as well as expected per-node storage. If the amount of resources used in the previous epoch exceeded a particular threshold (eg if a large number of blocks are more than half full), a number of new shards will be allocated, increasing the number of buckets and averaging down each of their expected usages. Adding new shards also means there will be more seats available for validation, which in turn brings more unique validators to the table as the per-seat price (in NEAR tokens) goes down. This computation is performed one epoch in advance of actually using these new shards so validators have the time they need to re-sync the necessary state and other shard information. Inflation The overall inflation (minting of new tokens) of the system is determined by how large the epoch reward is for running a validating node. The rate of minting of new tokens is capped at 5% per annum. The effective rate is computed per epoch (½ a day). It is calculated from the expected inflation rate per epoch minus the fees collected during the epoch. Each portion of fees captured by the platform is removed from inflation, thus reducing the overall inflation of the system as its usage increases. Should the system’s usage fees reliably exceed the tokens generated by the inflation, it will become deflationary overall. Because the system is sharded and has hidden validators whose assignment is unknown to the system, it must socialize all rewards. This means that the “security” fee is evenly allocated to all validators independent of how much transactions or storage their shards processed. Economic Stakeholders Validators: Provide the computational resource and security for the network by running nodes. Developers: Create the applications which run atop the network Token Holders: Accounts or applications which maintain token balances NEAR Foundation: An independent entity which coordinates the governance and technical evolution efforts of the network participants. Third Party Observers: The observers of the chain who provide extra fraud and bad behavior protection. Users: Users of applications on the network who do not maintain token balances. The impact of economic policies on each of these stakeholders is examined below. Validator Rewards https://near.org/papers/the-official-near-white-paper/ 18/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol As a “Proof of Stake” system, the NEAR platform is secure because the validators who run nodes put some of their tokens at stake as a kind of deposit to guarantee good behavior and for validator selection (prevent Sybil attack). Should these validators produce an invalid block or create an alternative chain (eg. with the goal of creating a double spend), they will be “slashed”, cutting this deposit. Validators are chosen based on the “Thresholded Proof of Stake” model which uses an auction to determine how many “seats” will be allocated to each prospective validator (by determining the minimum threshold number of tokens for a single seat). This auction is designed to provide fair (equal opportunity) allocation and allow as many people as possible to participate in the network’s validation process so it can achieve meaningful decentralization. A validator can deterministically expect to participate in the validation process in a proportion which matches their proportion of total stake in the network. A given validator may become one of several possible roles: 1. Block Producer 2. Chunk Producer 3. Hidden Validator Independent of the role the validator is assigned, its reward will be proportional to the percentage of total amount staked by the validator. This means there is no need to pool stake under a minimum required to become a validator. In exchange for servicing the network by producing blocks and chunks and providing security and data availability, validators are rewarded with target number of NEAR every epoch. The target value is computed in such a way that, on an annualized basis, it will be 4.5% of total supply. Because validators are selected on a per-epoch basis and they each need to do an equal amount of work to validate chunks, provide data availability and produce blocks, the reward gets allocated every epoch and gets divided proportionally to the stake of each participant. All transaction fees (minus the part which is allocated as the rebate for contracts) which are collected within each epoch are burned by the system. The inflationary reward is paid out to validators at the same rate regardless of the amount of fees collected or burned. Taken together, this means that system-wide inflation is reduced by an amount proportional to the amount of fees which are paid to the system and, should network usage fees exceed the system-wide inflation rate, the system will become deflationary. The computational requirements for running a validating node are designed to be minimal. Most operators should be able to do so easily with a standard cloud-hosted virtual machine, for example with Amazon AWS’s $100/month instance or a n1-highcpu-4 Google Cloud instance (and likely with even less robust hardware). For validators who anticipate staking substantial balances and thus who expect to participate in many shards simultaneously, they might want to utilize more hardware (and redundancy) https://near.org/papers/the-official-near-white-paper/ 19/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol to compensate for the extra storage, bandwidth and compute load they will need. Third Party Observers (“Hidden Validators” and “Fisherman”) A key disadvantage of an unsophisticated sharded blockchain system is that the overall system security is split because only a portion of the network’s validators are validating the transactions of a particular shard. NEAR employs two ways to counteract this problem: “Hidden Validators” and “Fishermen”. “Hidden Validators” are validators who are selected from the general validator pool and are assigned to validate shards that are unknown to any parties except themselves. This process, which is described in greater detail in other sections, ensures that it is extremely difficult to successfully corrupt a sufficient number of nodes to perfrom malicious behaviors in a shard. Hidden Validators are compensated for validating and signing off on the validity of chunks and blocks as a normal part of the validator compensation process. “Fishermen” are observing nodes who permissionlessly detect and report bad behavior. These nodes are synced with the network but are not necessarily participating in the consensus and don’t actually get paid for any specific ongoing activity. They can include wallet operators, application developers or exchange infrastructure. These nodes validate parts of the chain that are important to them and, if they detect issues, they can flag those issues via challenge. To prevent “griefing” of challenges, a small bond of 10 NEAR must be posted ahead of time. The system does not provide any reward for operating a node as a Fisherman (there is no reward for sending a successful challenge). Instead, participants who run a Fisherman node generally have outside motivations for maintaining the security of the network. Contract Rewards A portion of the fees generated by a particular transaction are provided back to the contracts that were run during that transaction. This “Contract Reward” may be distributed in accordance with the rules specified in the contract, for example it may be allocated to an account controlled by the contract developer, by investors, by a DAO, etc. The percentage of fees which are allocated to this reward is set to a minimum value as system-level parameter, initially 30%. Developers always can charge extra outside of this mechanism by requiring user to attach funds to the call. This creates a business model for developers who might otherwise not have a meaningful way of charging for their applications. Having a minimum fee which is set at the system level avoids a “race to the bottom” which results in zero rewards due to competition (or simply the “forking out” of the fee by another developer). This has a powerful effect of incentivizing developers to build applications and core contracts for the network because they will be directly compensated proportional to the usage of these contracts. https://near.org/papers/the-official-near-white-paper/ 20/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol Token Holders Token holders may choose not to participate in the staking process, for example because they are only temporary holders, they are providing liquidity for trading markets or they simply prefer not to participate. Token holders who do not participate in the staking and validation process receive no additional benefits from the operation of the network itself, though their tokens have utility by powering storage of the data and their usage of applications running on the network. Protocol Treasury To enable continues community growth and evolution of protocol, part of the inflation (10% of the inflation, or a total of 0.5% annually per initial parameters) is allocated to Protocol Treasury. In the future, these are expected to be managed by the community with the goal of coordinating long-term development of NEAR ecosystem, particularly efforts that won’t sustain themself like future protocol development. Given the current state of decentralized governance, initially treasury will be overseen by the NEAR Foundation. The NEAR Foundation’s mandate is to enable community-driven innovation to benefit people around the world. While the foundation is nonessential to the operation of the network, which is fully decentralized, it is already able to support the project’s ongoing evolution in ways that other entities would find difficult. For example, it is funding education, events or infrastructure projects which benefit the commons but do not have a particular business model and which would otherwise not be funded. The amount of inflation that is allocated to the Protocol Treasury also acts as a decentralizing economic force since it allows for the redistribution of capital back into the ecosystem to developers and other participants who support the commons who might not otherwise have stake to offer. Special Conditions Slashing Conditions and Progressive Slashing There are two major types of malicious behaviour possible on the NEAR platform: 1. Double Signing: Signing two or more different blocks at the same height. 2. Invalid Chunks: Signing a chunk with an invalid data or computational result. Malicious validators might double sign because they are trying to execute a chain reorganization which reverts certain transactions (which might allow them to conduct a “double spend” as a result). Non-malicious double signing in a Proof of Stake system can also happen due to a misconfiguration or issue in the software. https://near.org/papers/the-official-near-white-paper/ 21/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol To balance the risk of accidental slashing, NEAR uses “Progressive slashing”. This is where the portion of stake that is slashed is a multiple of the amount of stake that exhibited the double signing behavior during the epoch in question. This multiple is 3, so each malicious participant gets slashed by 3 * malicious_stake / total_satake. For an example of a double sign which leads to slashing, assume a validator has 1% of the total tokens which have been staked in a particular epoch. Assuming the total amount of tokens staked in the epoch is 50,000,000 NEAR, this validator has 500,000 NEAR at stake. If that validator double signs and there are no other double signs, the validator will lose 3% of their stake in that epoch, so they will have 485,000 of NEAR returned to them and 15,000 NEAR will be burned. If the total amount of stake that double signed within an epoch reaches 33% (which becomes dangerous for the network), the entire stake of all involved parties will be slashed. For an invalid chunk, the full stake of the validator gets slashed. This is because an invalid chunk is only possible if the node is actually malicious (they have modified the code). SECTION 07 Technology NEAR’s community-operated cloud uses a novel consensus algorithm and a scalable sharding architecture to achieve its high level design goals. The key elements of NEAR’s technology are: Sharding: The system is designed to scale horizontally and near-infinitely by distributing computation across multiple parallelized shards. Consensus: Consensus is achieved across all of the nodes which make up the network operators across all of the shards using the new Nightshade algorithm. Staking Selection and Game Theory: To participate in the validation process, stakers are selected using a secure randomized process which optimally distributes seats across parties and provides incentives for them to operate with good behavior. Randomness: NEAR’s randomness approach is unbiasable, unpredictable and can tolerate up to 1/3 of malicious actors before liveness is affected and 2/3 of malicious actors before any one can actually influence its output. Technology Design Principles https://near.org/papers/the-official-near-white-paper/ 22/45 08/11/2021, 12:10 Technology Design Principles The NEAR White Paper – NEAR Protocol NEAR’s overall system design principles are used to inform its technical design according to the following interpretations: 1. Usability: End users should be burdened with the lowest possible security obligations for a given type of interaction. Developers should be able to easily build, test and deploy contracts in familiar languages and should be able to provide their end-users with experiences close to today’s web. 2. Scalability: The platform should scale infinitely as its capacity is used. 3. Simplicity: The design of each of the system’s components should be as simple as possible in order to achieve their primary purpose. 4. Sustainable Decentralization: The barrier for participation in the platform as a validating node should be set as low as possible in order to bring a wide range of participants. Individual transactions made far in the future must be at least as secure as those made today in order to safeguard the value they modify. Summary NEAR focuses on providing solutions to the two core problems of today’s blockchains — Usability and Scalability. Usability for end users is achieved through offering a progressive security model for wallet interactions and by giving developers more opportunities to craft experiences which closely resemble the web today. These are provided by flexible and programmable key management implemented on the protocol level as a result of NEAR’s contract-based account model. This allows things like meta transactions, atomic account transfers, accounts with funds that are locked for specific usage and other account programmability and restriction use cases to be easily implemented. Usability for developers is provided by setting up the protocol to provide for browser-based debugging, familiar programming languages (like AssemblyScript and Rust) and contract usage rebates (“Contract Reward”). Scalability is provided by sharding the chain into a potentially unlimited number of subchains, each of which operates in parallel. Performance Characteristics and Tradeoffs One commonly referenced trilemma states that a system cannot achieve scalability, decentralization and security at the same time. NEAR’s sharding and validator selection approaches provide significant scalability and decentralization while mitigating tradeoffs in security that would normally occur with such improvements. https://near.org/papers/the-official-near-white-paper/ Another classic trilemma is posed by the CAP theorem, which states that a system can only achieve 2 of Consistency, Availability (aka “liveness”) and Partition Tolerance. Given that partition tolerance cannotbesacrificedinthiscasethetradeoffisreallybetweenconsistencyandavailability 23/45 08/11/2021, 12:10 The NEAR White Paper – NEAR Protocol cannot be sacrificed in this case, the tradeoff is really between consistency and availability. In blockchain-based systems, an illustrative example is what happens if the network splits into two parts for a week. A *consistent* system will completely shut down one (or both) of the halves until the network is restored so that the two parts do not become inconsistent. An *available* system (like Bitcoin) will continue to run both halves of the network independently and, when they are restored to unity, the operations of one half will be wiped out in favor of the operations of the other. NEAR currently favors availability at a system level but individual users can choose to not accept blocks without >50% signature thresholds as a way of locally requiring consistency as well. The performance of the system will be highly dependent on the types of transactions which are processed and the actual hardware which is supporting it. For simple financial transactions, per-shard throughput could range from 400-2000 transactions per second. Sharding Current approaches to scalability typically fall into two categories: 1. Vertical Scaling: Achieved by improving the performance of the existing hardware of a system. In the case of blockchain-based systems, it typically means running a network containing fewer nodes that each require *better* hardware. This creates an initial improvement in throughput while limiting future improvements to roughly the rate of increase in performance of computing hardware (often considered to be “Moore’s Law”). This leaves the network without the ability to scale at a rate commensurate with its adoption. 2. Horizontal Scaling: Achieved by adding *more* hardware to a system. In the case of blockchains, this is typically done by ensuring that an increase in the number of nodes participating in the network improves the performance of that network by a commensurate amount, for example by parallelizing computation across multiple “shards”. NEAR uses a sharding approach to provide scalability horizontally, which allows it to scale capacity alongside increases in demand. Cross-Chain and Cross-Shard Communication One of the biggest difficulties with any form of cross-chain communication, whether this occurs within the shards of a single chain or across multiple chains, is determining that an incoming transaction from another chain is valid. There are 3 approaches for validating cross-chain transactions: 1. Dual Validation: Have the validators for the receiving chain also validate on the sending chain. This is used by Quarkchain. It has the downside that validators do not scale well in this approach. 2. Trust the Transaction: Assume that if a transaction has been received, it must be valid. In Cosmos, for example, a transaction that is copied to the main hub is considered irreversible. They keep track of the total number of tokens in each economy so you cannot create new ones but you could theoretically create invalid transfers between parties (eg steal tokens from other parties). https://near.org/papers/the-official-near-white-paper/ 24/45 08/11/2021, 12:10 y The NEAR White Paper – NEAR Protocol p g p 3. Beacon Chain w/ Rollback: A beacon chain verifies the state transitions all of the other chains using a small subset of validators and, if a problem is detected, all chains are rolled back. To achieve atomicity, this reversion should happen, though it should happen only rarely and should be immediately detected. NEAR focuses on the 3rd approach. With the assumption that an adaptive adversary cannot corrupt the validators of a shard within a day, validators of each shard can be rotated daily to help add a layer of security. But presumably it is possible (if very difficult) for an adaptive adversary to corrupt a shard’s validators within a given day. To help offset this, other protocols use a smaller committee which rotates far more rapidly (eg every few minutes) and validates across shards. In order for this smaller committee to perform their validations without having to download the entire state of each shard (which cannot be done in this timeframe), they receive only that portion of the state which was actually affected. But it is difficult to send the state with each change — a single transaction might affect 100mb of state at a time. This is where the Nightshade sharding approach comes in. Nightshade Nightshade modifies the typical sharding abstraction and assumes that all of the shards combine together to produce a single block. This block is produced with a regular cadence regardless of whether each individual shard has produced its “chunk” for that specific block height. So every chunk for each shard will either be present or not. There must be a fork choice rule to decide on the proper fork. This is still under development but will most likely resemble LMD Ghost. It will include the weight of how many validator attestations have been received for a given chunk and block. There is a single validator assigned to produce each block. This validator must assemble the chunks which are provided to it during that block’s time period into the period’s block. The assignment of this validator will rotate through the existing validator set (eg 100 validators). This leader does not accept transactions, only chunks. For each individual shard and period, a single validator is assigned to produce its chunk. If that validator is not present, the shard will stall for that period. Each shard has its own smaller pool of validators which is pulled from the main pool. The shard leader position rotates among this smaller pool (eg 4 validators) in the same way that the overall block leader is selected. Thus, if a single validator is absent and the shard chunk stalls for one period, the next validator will likely be present to continue the chain’s operation in the following period. Learn more about NEAR’s sharding design in the Nightshade paper. HiddenValidators https://near.org/papers/the-official-near-white-paper/ 25/45 08/11/2021, 12:10 Hidden Validators The NEAR White Paper – NEAR Protocol In order to provide additional security, NEAR uses Hidden Validators. These are a smaller committee for each shard (on average 100 validators) who verify each chunk. Rather than having this assignment be on the blockchain and thus publicly visible to all participants, the validators themselves figure out their assignment individually by drawing a set of shard ids from a Verifiable Random Function (VRF). This way each individual validator is aware of which shards they must verify but, to corrupt them, an adversary must bribe a large percentage of total validators across all shards to reveal their masks. Further, the number of hidden validators assigned to a particular block is randomly determined as well. This prevents an adversary from knowing exactly how many hidden validators they even need to corrupt in the first place in order to successfully pull off an attack. This prevents attacks where an adversary broadcasts their intent and waits for the fishermen to come to them (revealing which shards they are validating). Due to the nature of the verification, any single hidden validator can present a proof that the chunk is invalid, a so-called “fraud proof”. The selection of the smaller per-shard committee is done for every epoch (½ day) from the same pool as the block and chunk producers, which is the total set of nodes which staked. For example, if there are 100 seats per shard and 100 shards, there are a total of 10,000 seats. 100 of them will be allocated to be the chunk producers and the rest will be hidden validators. Fishermen In addition to the hidden validators who are assigned to provide security for each shard, any other node operator can participate permissionlessly as a so-called “fisherman.” This third-party node can provide the same fraud proof as a hidden validator and thus they too can kick off the process of slashing and rollback. This means that, even if an adversary successfully corrupted the entire hidden validator pool, they have no guarantee that their efforts will not be discovered by one of these independent fishermen and are thus highly disincentivized. Preventing Lazy Validators One potential problem with validators is that they can be “lazy”. After every block, a validator must receive the new chunk, download the new state and run validation on that block. They could, however, choose to do nothing unless they see another validator submitting a fraud proof and only then do they actually validate the latest block and try to submit a proof of their own. Thus a chain can end up paying validators but receive no meaningful work from them. This is mitigated by making validators to first commit to their decision (if chunk is valid or invalid) and then reveal what they committed. This creates an incentive to do proper work because they have value atstakeandwillbeslashediftheymissaninvalidchunksfasdf https://near.org/papers/the-official-near-white-paper/ 26/45