# Introducing the Beta Version of NEAR Lake Framework: Simplifying Data Processing on NEAR Protocol We're thrilled to announce the release of the beta version of the NEAR Lake Framework, a significant update that brings enhanced functionality and simplicity to data processing on the NEAR Protocol. With a focus on high-level abstractions and ease of use, this beta version marks a major milestone in our mission to streamline decentralized data integration and analysis. In this post, we'll explore the key features and improvements introduced in this release, empowering developers to leverage the full potential of the NEAR Lake Framework. ## Streamlined High-Level Structure: In response to developer feedback and extensive analysis, we've made substantial improvements to the NEAR Lake Framework. The latest release, version 0.8.0 for the Rust language, presents a more high-level approach, aiming to simplify the complexities associated with the `StreamerMessage` structure. By abstracting away low-level details, we're empowering developers to focus on building data applications without getting bogged down in implementation intricacies. We're also excited to announce that a TypeScript version is in the pipeline, with an expected release in the near future. ## NEAR Lake Primitives: To further enhance the developer experience, we've introduced NEAR Lake Primitives, a dedicated crate that encapsulates all the new high-level primitives. While documentation is currently a work in progress, we've decided to release the beta version to allow developers to explore and experiment with these powerful tools. Rest assured, we're committed to refining and perfecting the documentation to provide comprehensive guidance as we move towards the official release. ## Simplified Starting Boilerplate: Starting a new project with the NEAR Lake Framework is now easier than ever. We've significantly simplified the initial boilerplate, aligning it with the TypeScript version. Developers simply need to create an indexing function that receives blocks and handles them based on their specific goals. Let's take a look at a simplified example: ```rust fn main() -> anyhow::Result<()> { near_lake_framework::LakeBuilder::default() .testnet() .start_block_height(112205773) .build()? .run(handle_block)?; Ok(()) } // The handler function to take the `Block` // and print the block height async fn handle_block( block: near_lake_primitives::block::Block, ) -> anyhow::Result<()> { eprintln!( "Block #{}", block.block_height(), ); Ok(()) } ``` ## Embracing NEAR Protocol Fundamentals: In this version, we've revamped the fundamentals of the NEAR Lake Framework, allowing indexer developers to focus on high-level entities derived from the NEAR Protocol: Actions and Events. This shift enables developers to build data pipelines that align closely with the core concepts of the NEAR Protocol. Consider the following example of an indexer that processes actions: ```rust async fn print_function_calls_to_my_account( mut block: near_lake_primitives::block::Block, ) -> anyhow::Result<()> { let block_height = block.block_height(); let actions: Vec<&near_lake_primitives::actions::FunctionCall> = block .actions() .filter(|action| action.receiver_id().as_str() == "social.near") .filter_map(|action| action.as_function_call()) .collect(); if !actions.is_empty() { println!("Block #{:?}\n{:#?}", block_height, actions); } Ok(()) } ``` ## Introducing Contexts: To enable even more flexibility, we've introduced the concept of Contexts within the NEAR Lake Framework. Developers can now pass a context to each call of their indexing function, opening up new possibilities for customization and collaboration. Notably, we've implemented mechanisms to facilitate seamless sharing of contexts between developers. Let's examine an example of using the context: ```rust #[derive(LakeContext)] struct MyContext { my_field: String, } fn main() -> anyhow::Result<()> { let context = MyContext { my_field: "my_value".to_string(), }; near_lake_framework::LakeBuilder::default() .testnet() .start_block_height(112205773) .build()? .run_with_context(handle_block, &context)?; Ok(()) } ``` ## NEAR Lake Parent Transaction Cache context: As part of our commitment to delivering practical solutions, we're excited to announce the release of the NEAR Lake Parent Transaction Cache. This shared context enables developers to easily build and maintain a cache of parent transactions for receipts observed by the indexer. It's a ready-to-use solution that enhances efficiency and reduces redundancy. For more information and examples, please visit [NEAR Lake Parent Transaction Cache](https://github.com/near/near-lake-framework-rs/tree/main/lake-parent-transaction-cache). ## Your Feedback Matters: We value the feedback and insights of the developer community. While we anticipate rapid adoption of the beta version, we understand that migrating existing logic might be inconvenient for some. Therefore, we assure you that we will continue supporting the 0.7.x version and keep it updated with nearcore primitives. We'll provide a clear timeline for the maintenance end date in the coming weeks. ## Conclusion: The beta version of the NEAR Lake Framework represents a significant leap forward in simplifying data processing on the NEAR Protocol. With streamlined high-level structures, improved starting boilerplate, the introduction of contexts, and powerful primitives, developers now have a robust toolset to build decentralized data applications more efficiently than ever before. We invite you to explore the beta version, provide feedback, and join us on this exciting journey towards a more accessible and powerful data ecosystem on NEAR Protocol. ### Links: - [NEAR Lake Framework beta](https://crates.io/crates/near-lake-framework/0.8.0-beta.2) - [NEAR Lake Primitives](https://crates.io/crates/near-lake-primitives/0.8.0-beta.2) - [NEAR Lake Parent Transaction Cache Context](https://crates.io/crates/near-lake-parent-transaction-cache/0.8.0-beta.2) - [Feedback discussion post](https://github.com/near/near-lake-framework-rs/discussions/84)