# Ethereum Swarm Documentation > Swarm is a decentralised storage and communication system for a sovereign digital society. This file contains all documentation content in a single document following the llmstxt.org standard. ## DISC DISC (Distributed Immutable Store for Chunks) is a storage solution developed by Swarm based on a modified implementation of a [Kademlia DHT](./kademlia.mdx) which has been specialized for data storage. Swarm's implementation of a DHT differs significantly in that it stores the content in the DHT directly, rather than just storing a list of seeders who are able to serve the content. This approach allows for much faster and more efficient retrieval of data. ## Kademlia Topology and Routing [Kademlia](./kademlia.mdx) is a distributed hash table (DHT) widely used in peer-to-peer networks such as Ethereum and Bittorent. It serves as the routing and topology foundation for communication between nodes in the Swarm network. It organizes nodes based on their overlay addresses and ensures that messages are relayed efficiently, even in a dynamic, decentralized environment. One of the advantages of using Kademlia as a model for network topology is that both the number of forwarding "hops" required to route a chunk to its destination and the number of peer connections required to maintain Kademlia topology are logarithmic to the size of the network (a minimum of two connections is required in order to maintain Kademlia topology in case of network churn - nodes dropping in and out of the network). This makes Swarm a highly scalable system which is efficient even at very large scales. ## Neighborhoods [Neighborhoods](./neighborhoods.md) are groups of nodes which are responsible for sharing the same chunks. The chunks which each neighborhood is responsible for storing are defined by the proximity order of the nodes and the chunks. In other words, each node is responsible for storing chunks with which their overlay addresses share a certain number of prefix bits, and together with other nodes which share the same prefix bits, make up neighborhoods which share the responsibility for storing the same chunks. Neighborhoods play a key role in providing data redundancy for chunks stored on Swarm since each node in a neighborhood will keep copies of the same chunks. The optional [erasure coding](./erasure-coding.md) feature can also be enabled for added redundancy and greater data protection. ## Chunks In the DISC model, **chunks** are the basic storage unit of the network layer. When a file is uploaded to Swarm, it gets broken down into chunks - pieces of at most 4KB, each with a small metadata header. Every chunk gets its own address, and each chunk is stored by the nodes whose [overlay addresses](./../../references/glossary.md#overlay) are closest to that chunk address. Chunk addressing is *deterministic, collision-free, and uniformly distributed,* which is what gives Swarm local validity, integrity guarantee, and load balancing across nodes. There are two fundamental chunk types: content-addressed chunks and single-owner chunks. ### Content-Addressed Chunks and Single-Owner Chunks Content-addressed chunks are chunks whose address is based on the hash digest of their data. Using a hash as the chunk address makes it possible to verify the integrity of chunk data. Swarm uses the BMT hash function, a binary Merkle tree (BMT) built with Keccak256 over the 32-byte segments of the chunk data; payloads shorter than 4KB are hashed as if zero-padded up to 4KB. A content-addressed chunk has an at most 4KB payload, and its address is calculated as the hash of the 8-byte span and the BMT root of the payload. Because the address is derived from the content, a content-addressed chunk cannot be changed in place. Different content yields a different address. Source: The Book of Swarm - Figure 2.7 - "Content addressed chunk" For single-owner chunks on the other hand, the address is calculated as the hash of an identifier and an owner's Ethereum address. The content consists of an arbitrary data payload along with required headers: a 32-byte identifier, a 65-byte signature, and the same 8-byte span used by content-addressed chunks. The signature signs off on the identifier and the BMT hash of the span and payload, so integrity comes from the owner's signature rather than from the content itself. Validating a single-owner chunk means recovering the owner's address from the signature and checking the hash of the identifier and that address against the chunk address, which is in effect, an authentication that the owner has write access to that address. Source: The Book of Swarm - Figure 2.10 - "Single owner chunk" Single-owner chunks form the basis for feeds. A feed chunk is a single-owner chunk whose identifier is the hash of a feed topic and an index, so each update is published as a new chunk at a new, deterministically derivable address. A reader can therefore find the latest update from the owner's address and the topic alone, even though no individual chunk is ever overwritten, meaning the chunk store itself remains immutable. | | Content-addressed chunk (CAC) | Single-owner chunk (SOC) | |---|---|---| | Address derived from | Hash of the 8-byte span and the BMT root of the payload | Hash of the 32-byte identifier and the owner's Ethereum address | | Integrity attested by | The content itself — the address is the hash of the content | The owner's signature over the identifier and the BMT hash of span and payload | | Chunk mutable? | No — different content yields a different address | No — signing a second payload for the same identifier makes network behaviour unpredictable; mutability is achieved at the feed layer | | Max payload | 4 KB + 8-byte span header | 4 KB + 105 bytes of headers (identifier, signature, span) | | Basis for | File and manifest data | Feeds (a mutable resource resolved from owner address + topic) | ## Push-Sync, Pull-Sync, and Retrieval Protocols When a file is first uploaded to Swarm, it gets broken down by the uploading Bee node chunks which are then distributed amongst other Bee nodes in the Swarm network. Chunks get distributed to the target neighborhood by the ***push-sync*** protocol. Once a chunk reaches its destination, it will then be duplicated and synced to other nodes in order to achieve data redundancy through the ***pull-sync*** protocol. The pull-sync protocol operates continuously as nodes enter or exit the network – ensuring that data redundancy is always maintained. When a client node requests a file for download, its request gets forwarded by the ***retrieval-protocol*** to all the nodes storing the relevant chunks, and then those chunks get returned to the requesting node and the file gets reconstructed from its constituent chunks. --- ## Erasure Coding Erasure coding (also known as erasure code) is an efficient and flexible approach to data protection which is an optional feature for Swarm uploads. It is a technique that increases data protection by enabling the recovery of original data even when some encoded chunks are lost or corrupted. When used, it ensures that data on Swarm can always be accessed reliably, even if some nodes or entire neighborhoods go offline. Refer to the [official erasure coding paper](https://papers.ethswarm.org/p/erasure/) for more in depth details. ## How does erasure coding work? {#how-it-works} Erasure coding enhances data protection by dividing the source data into "chunks" and adding additional redundant chunks. Specifically, data is divided into **m** chunks, and **k** additional chunks are generated, resulting in **m + k** total chunks. The data is encoded across these chunks such that as long as **m** chunks are intact, the original data can be fully reconstructed. Chunks are then distributed across the network as with a standard upload. This approach provides a robust method for data recovery in distributed storage networks like Swarm. ### Example For an 8KB image, if we set **m = 2** and **k = 1**, we create 3 chunks (2 original + 1 redundant). As long as any 2 of these 3 chunks are available, we can reconstruct the original data. By increasing **k** to 4, we can tolerate the loss of up to 4 chunks while still recovering the original data. ### Levels of Protection In Swarm's implementation of erasure coding, there are five levels of protection, None, Medium, Strong, Insane, and Paranoid. For each level, the **m** and **k** values have been adjusted in order to meet a certain level of data protection: ***Table A:*** | Redundancy Level Value | Level Name | Chunk Loss Tolerance | | ---------------- | --------- | ----------------------------------- | | 0 | None | 0% | | 1 | Medium | 1% | | 2 | Strong | 5% | | 3 | Insane | 10% | | 4 | Paranoid | 50% | The "Redundancy Level" is a numeric value for each level of protection, the "Level Name" is the official name for each level, and the "Chunk Loss Tolerance" column corresponds to the exact level of data protection for each level. For each redundancy level, the original data is retrievable with >=99.9999% statistical certainty given a percent chunk loss equal or less than the percent shown in the "Chunk Loss Tolerance" column. Note that this guarantee of retrievability is for each 128 chunk segment, and therefore does not correspond to retrievability of a whole file. The retrievability failure rate for any individual file depends on the size of the file, and increases with the size of the file. For a detailed explanation of how to calculate the retrievability of any sized file refer to [section 3 in the erasure coding paper](https://papers.ethswarm.org/erasure-coding.pdf). ## Usage For usage instructions, see the [erasure coding page in the "Develop" section](./../../develop/tools-and-features/erasure-coding.md). ## Cost Calculation In Swarm's implementation of erasure coding, there are five levels of protection: None, Medium, Strong, Insane, and Paranoid. Each level adds additional parity chunks for a corresponding increase in data protection (and also cost). The table below shows the number of parities and data chunks for each level, as well as the percent increase in cost vs a non-erasure coded upload. ***Table B:*** | Redundancy | Parities | Data Chunks | Percent | Chunks Encrypted | Percent Encrypted | |----------|----------|--------|---------|------------------|-------------------| | None | 0 | 128 | *0%* | 64 | 0% | | Medium | 9 | 119 | *7.6%* | 59 | 15.3% | | Strong | 21 | 107 | *19.6%* | 53 | 39.6% | | Insane | 31 | 97 | 32% | 48 | 64.6% | | Paranoid | 90 | 38 | 236.8% | 19 | 473.7% | For each redundancy level, there are **m + k** = 128 chunks, where **m** are the data chunks (shown in column "Data Chunks") and **k** are the parity chunks (shown in column "Parities"). The "Percent" and "Percent Encrypted" columns show percent of "parity overhead" cost increase from using erasure coding for normal and encrypted uploads respectively. ### Cost Calculation for Smaller Uploads To find the percent increase in cost for uploads of less than 128 chunks, refer to the table below: ***Table C:*** | Security | Parities | Chunks | Percent | Chunks Encrypted | Percent Encrypted | |----------|----------|--------|-------------|------------------|-------------------| | Medium | 2 | 1 | 200% | | | | Medium | 3 | 2-5 | 150% - 60% | 1-2 | 300% - 150% | | Medium | 4 | 6-14 | 66.7% - 28.6%| 3-7 | 133.3% - 57.1% | | Medium | 5 | 15-28 | 33.3% - 17.9%| 7-14 | 71.4% - 35.7% | | Medium | 6 | 29-46 | 20.7% - 13% | 14-23 | 42.9% - 26.1% | | Medium | 7 | 47-68 | 14.9% - 10.3%| 23-34 | 30.4% - 20.6% | | Medium | 8 | 69-94 | 11.6% - 8.5%| 34-47 | 23.5% - 17% | | Medium | 9 | 95-119 | 9.5% - 7.6% | 47-59 | 19.1% - 15.3% | | Strong | 4 | 1 | 400% | | | | Strong | 5 | 2-3 | 250% - 166.7%| 1 | 500% | | Strong | 6 | 4-6 | 150% - 100% | 2-3 | 300% - 200% | | Strong | 7 | 7-10 | 100% - 70% | 3-5 | 233.3% - 140% | | Strong | 8 | 11-15 | 72.7% - 53.3%| 5-7 | 160% - 114.3% | | Strong | 9 | 16-20 | 56.2% - 45% | 8-10 | 112.5% - 90% | | Strong | 10 | 21-26 | 47.6% - 38.5%| 10-13 | 100% - 76.9% | | Strong | 11 | 27-32 | 40.7% - 34.4%| 13-16 | 84.6% - 68.8% | | Strong | 12 | 33-39 | 36.4% - 30.8%| 16-19 | 75% - 63.2% | | Strong | 13 | 40-46 | 32.5% - 28.3%| 20-23 | 65% - 56.5% | | Strong | 14 | 47-53 | 29.8% - 26.4%| 23-26 | 60.9% - 53.8% | | Strong | 15 | 54-61 | 27.8% - 24.6%| 27-30 | 55.6% - 50% | | Strong | 16 | 62-69 | 25.8% - 23.2%| 31-34 | 51.6% - 47.1% | | Strong | 17 | 70-77 | 24.3% - 22.1%| 35-38 | 48.6% - 44.7% | | Strong | 18 | 78-86 | 23.1% - 20.9%| 39-43 | 46.2% - 41.9% | | Strong | 19 | 87-95 | 21.8% - 20% | 43-47 | 44.2% - 40.4% | | Strong | 20 | 96-104 | 20.8% - 19.2%| 48-52 | 41.7% - 38.5% | | Strong | 21 | 105-107| 20% - 19.6% | 52-53 | 40.4% - 39.6% | | Insane | 5 | 1 | 500% | | | | Insane | 6 | 2 | 300% | 1 | 600% | | Insane | 7 | 3 | 233.3% | 1 | 700% | | Insane | 8 | 4-5 | 200% - 160% | 2 | 400% | | Insane | 9 | 6-8 | 150% - 112.5%| 3-4 | 300% - 225% | | Insane | 10 | 9-10 | 111.1% - 100%| 4-5 | 250% - 200% | | Insane | 11 | 11-13 | 100% - 84.6%| 5-6 | 220% - 183.3% | | Insane | 12 | 14-16 | 85.7% - 75% | 7-8 | 171.4% - 150% | | Insane | 13 | 17-19 | 76.5% - 68.4%| 8-9 | 162.5% - 144.4% | | Insane | 14 | 20-22 | 70% - 63.6% | 10-11 | 140% - 127.3% | | Insane | 15 | 23-26 | 65.2% - 57.7%| 11-13 | 136.4% - 115.4% | | Insane | 16 | 27-29 | 59.3% - 55.2%| 13-14 | 123.1% - 114.3% | | Insane | 17 | 30-33 | 56.7% - 51.5%| 15-16 | 113.3% - 106.2% | | Insane | 18 | 34-37 | 52.9% - 48.6%| 17-18 | 105.9% - 100% | | Insane | 19 | 38-41 | 50% - 46.3% | 19-20 | 100% - 95% | | Insane | 20 | 42-45 | 47.6% - 44.4%| 21-22 | 95.2% - 90.9% | | Insane | 21 | 46-50 | 45.7% - 42% | 23-25 | 91.3% - 84% | | Insane | 22 | 51-54 | 43.1% - 40.7%| 25-27 | 88% - 81.5% | | Insane | 23 | 55-59 | 41.8% - 39% | 27-29 | 85.2% - 79.3% | | Insane | 24 | 60-63 | 40% - 38.1% | 30-31 | 80% - 77.4% | | Insane | 25 | 64-68 | 39.1% - 36.8%| 32-34 | 78.1% - 73.5% | | Insane | 26 | 69-73 | 37.7% - 35.6%| 34-36 | 76.5% - 72.2% | | Insane | 27 | 74-77 | 36.5% - 35.1%| 37-38 | 73% - 71.1% | | Insane | 28 | 78-82 | 35.9% - 34.1%| 39-41 | 71.8% - 68.3% | | Insane | 29 | 83-87 | 34.9% - 33.3%| 41-43 | 70.7% - 67.4% | | Insane | 30 | 88-92 | 34.1% - 32.6%| 44-46 | 68.2% - 65.2% | | Insane | 31 | 93-97 | 33.3% - 32% | 46-48 | 67.4% - 64.6% | | Paranoid | 19 | 1 | 1900% | | | | Paranoid | 23 | 2 | 1150% | 1 | 2300% | | Paranoid | 26 | 3 | 866.7% | 1 | 2600% | | Paranoid | 29 | 4 | 725% | 2 | 1450% | | Paranoid | 31 | 5 | 620% | 2 | 1550% | | Paranoid | 34 | 6 | 566.7% | 3 | 1133.3% | | Paranoid | 36 | 7 | 514.3% | 3 | 1200% | | Paranoid | 38 | 8 | 475% | 4 | 950% | | Paranoid | 40 | 9 | 444.4% | 4 | 1000% | | Paranoid | 43 | 10 | 430% | 5 | 860% | | Paranoid | 45 | 11 | 409.1% | 5 | 900% | | Paranoid | 47 | 12 | 391.7% | 6 | 783.3% | | Paranoid | 48 | 13 | 369.2% | 6 | 800% | | Paranoid | 50 | 14 | 357.1% | 7 | 714.3% | | Paranoid | 52 | 15 | 346.7% | 7 | 742.9% | | Paranoid | 54 | 16 | 337.5% | 8 | 675% | | Paranoid | 56 | 17 | 329.4% | 8 | 700% | | Paranoid | 58 | 18 | 322.2% | 9 | 644.4% | | Paranoid | 59 | 19 | 310.5% | 9 | 655.6% | | Paranoid | 61 | 20 | 305% | 10 | 610% | | Paranoid | 63 | 21 | 300% | 10 | 630% | | Paranoid | 65 | 22 | 295.5% | 11 | 590.9% | | Paranoid | 66 | 23 | 287% | 11 | 600% | | Paranoid | 68 | 24 | 283.3% | 12 | 566.7% | | Paranoid | 70 | 25 | 280% | 12 | 583.3% | | Paranoid | 71 | 26 | 273.1% | 13 | 546.2% | | Paranoid | 73 | 27 | 270.4% | 13 | 561.5% | | Paranoid | 75 | 28 | 267.9% | 14 | 535.7% | | Paranoid | 76 | 29 | 262.1% | 14 | 542.9% | | Paranoid | 78 | 30 | 260% | 15 | 520% | | Paranoid | 80 | 31 | 258.1% | 15 | 533.3% | | Paranoid | 81 | 32 | 253.1% | 16 | 506.2% | | Paranoid | 83 | 33 | 251.5% | 16 | 518.8% | | Paranoid | 84 | 34 | 247.1% | 17 | 494.1% | | Paranoid | 86 | 35 | 245.7% | 17 | 505.9% | | Paranoid | 87 | 36 | 241.7% | 18 | 483.3% | | Paranoid | 89 | 37 | 240.5% | 18 | 494.4% | | Paranoid | 90 | 38 | 236.8% | 19 | 473.7% | ### Example Cost Calculation For each redundancy level, there are m + k = 128 chunks, where m are the data chunks (shown in column "Data Chunks") and k are the parity chunks. If the number of chunks in the data being uploaded are an exact multiple of m, then the percent cost of the upload will simply equal the one shown in table B from the section above in the "Percent" column for the corresponding redundancy level. #### Exact Multiples For example, if we are uploading with the Strong redundancy level, and our source data consists of 321 (3 * 107) chunks, then we can simply use the percentage from the "Percent" column for the Strong level - 19.6% (63 parities / 321 data chunks). #### With Remainders However, generally speaking uploads will not come in exact multiples of m, so we need to adjust our calculations. To do so we need to use table C from the section above which shows the number of parities for sets of chunks starting at a single chunk for each redundancy level up to the maximum number of data chunks for that level. Then we simply sum up the total parities and data chunks for the entire upload and calculate the resulting percentage. Let's say for example we have a source file of 340 chunks which we want to upload with the Strong level of protection. Referring to table B, we see for the Strong level there are 21 parity chunks for each 107 data chunks. 340 / 107 = ~3.177, meaning our upload will have three full sets of 128 chunks where m = 107 and k = 21. The remainder can be calculated from the modulus of 340 % 107 = 19 Looking at our chart, we can see that at the Strong level for 19 data chunks we need 9 parity chunks. From this we can calculate the final percentage price: 72 / 340 = 21.17%. --- ## Kademlia Kademlia is a distributed hash table (DHT) algorithm used in peer-to-peer networks to efficiently store and retrieve data without relying on centralized servers. It organizes nodes into an overlay network that ensures efficient routing using a binary tree structure. ## Kademlia Key Concepts ### **XOR Distance Metric** Kademlia uses a distance metric based on the XOR (exclusive OR) between any addresses. This allows nodes to calculate "distance" from each other. Lookups are made by recursively querying nodes that are progressively closer to the target. ### **Routing Table** Each node in a Kademlia network maintains a routing table containing information about other nodes, organized by the XOR distance between node IDs. ## Kademlia Advantages ### **Efficient Lookups** To retrieve a specific chunk, a node uses Kademlia's lookup process to find and fetch the chunk from a node in the neighborhood where it is stored. The number of hops required for a chunk to be retrieved is logarithmic to the number of nodes in the network, meaning lookups remain efficient even as the network grows larger and larger. ### **Fault Tolerance** Because nodes' peer lists are regularly refreshed through lookups and interactions, and because redundant copies of data are replicated within the network, the network remains functional even when individual nodes leave or fail. ### **Scalability** Kademlia's design allows it to scale to large networks, as each node only needs to keep track of a small subset of the total nodes in the network. The required set of connected peers grows logarithmically with the number of nodes, making it efficient even in large networks. ## Kademlia in Swarm Swarm's version of Kademlia differs from commonly used implementations of Kademlia in several important ways: ### Proximity Order & Neighborhoods Swarm introduces the concept of [proximity order (PO)](./../../references/glossary.md#proximity-order-po) as a discrete measure of node relatedness between two addresses. In contrast with Kademlia distance which is an exact measure of relatedness, PO is used to measure the relatedness between two addresses on a discrete scale based on the number of shared leading bits. Since this metric ignores all the bits after the shared leading bits, it is not an exact measure of distance between any two addresses. In Swarm's version of Kademlia, nodes are grouped into [neighborhoods](./neighborhoods.md) of nodes based on PO (ie., neighborhood are composed of nodes which all share the same leading binary prefix bits). Each neighborhood of nodes is responsible for storing the same set of chunks. Neighborhoods are important for ensuring data redundancy, and they also play a role in the incentives system which guarantees nodes are rewarded for contributing resources to the network. ### Forwarding Kademlia Kademlia comes in two flavors, iterative and forwarding. In iterative Kademlia, the requesting node directly queries each node it contacts for nodes that are progressively closer to the target until the node with the requested chunk is found. The chunk is then sent directly from the storer node to the node which initiated the request. In contrast, Swarm makes use of forwarding Kademlia. Here each node forwards the query to the next closest node in the network, and this process continues until a node with the requested chunk is found. Once the chunk is found, it is sent back along the same chain of nodes rather than sent directly to the initiator of the request. The main advantage of forwarding Kademlia is that it maintains the anonymity of the node which initiated the request. | | Iterative Kademlia | Forwarding Kademlia (Swarm) | |---|---|---| | Who queries the next node | The requesting node, directly | Each intermediate node forwards the query | | Chunk return path | Sent directly from storer to requester | Relayed back along the same chain of nodes | | Requester anonymity | No | Yes | Source: The Book of Swarm - Figure 2.3 - "Iterative and Forwarding Kademlia routing" ### Neighborhood Based Storage Incentives Swarm introduces a storage incentives layer on top of its Kademlia implementation in order to reward nodes for continuing to provide resources to the network. Neighborhoods play a key role in the storage incentives mechanism. Storage incentives take the role of a "game" in which nodes play to win a reward for storing the correct data. Each round in the game, one neighborhood is chosen to play, and all nodes within the same neighborhood participate as a group. The nodes each compare the data they are storing with each other to make sure they are all storing the data they are responsible for, and one node is chosen to win from among the group. You can read more about how storage incentives work in the dedicated page for storage incentives. --- ## Neighborhoods In Swarm, a neighborhood refers to an area of responsibility within the network, where nodes in proximity to one another share the task of storing and maintaining data chunks. Nodes within a neighborhood replicate chunks to ensure that if one node goes offline, other nodes in the neighborhood can still retrieve and serve the content. :::info To see current neighborhood populations and the current storage depth / storage radius navigate to the ["Neighborhoods" page of Swarmscan.io](https://swarmscan.io/neighborhoods). The terms "depth" and "radius" are often used interchangeably when discussing neighborhoods. Both refer to number of shared leading bits of node and chunk addresses used to determine the nodes and chunks which fall into which neighborhoods. ::: ## Key Concepts ### Proximity Order (PO) The PO measures how close a node is to a particular chunk of data or another node. It is defined as the number of shared leading bits between two addresses. Proximity order plays a role in how neighborhoods are defined, as a node’s neighborhood extends up to its storage depth, covering all nodes within that proximity​. ### Reserve Depth The reserve depth is the shallowest PO at which neighborhoods are able to store all of the chunks which have been paid for through [postage stamp batch](./../incentives/overview.mdx#postage-stamps) purchases. ### Storage Depth Storage depth is the shallowest PO at which neighborhoods are able to store all the chunks which have been *uploaded*. If 100% of all all chunks which have been paid for have been stamped and uploaded to the network, then storage depth will equal reserve depth. However, it is common that stamp batches are not always fully utilized, meaning that it is possible for the storage depth to be shallower than the reserve depth. Storage depth is the proximity order of chunks for which a node must synchronize and store chunks, and it is determined by nodes' reserve sizes in combination with the amount of chunks actually uploaded. ### Neighborhood Depth Neighborhood depth for a node is the highest (deepest) PO *`d`* where the node has at least 3 peers which share the same *`d`* number of leading binary prefix bits in their addresses. ### Neighborhood A neighborhood is a set of nodes in close proximity to each other based on their proximity order (PO). Each node within a storage-depth-defined neighborhood interacts with other nodes to store and replicate data chunks, ensuring availability and redundancy. ## Example neighborhood Let's take a closer look at an example. Below is a neighborhood of six nodes at depth 10. Each node is identified by its Swarm address, which is a 256 bit hexadecimal number derived from the node's Gnosis Chain address, the Swarm network id, and a random nonce. > da4cb0d125bba638def55c0061b00d7c01ed4033fa193d6e53a67183c5488d73 > da5d39a5508fadf66c8665d5e51617f0e9e5fd501e429c38471b861f104c1504 > da7a974149543df1b459831286b42b302f22393a20e9b3dd9a7bb5a7aa5af263 > da76f8fccc3267b589d822f1c601b21b525fdc2598df97856191f9063029d21e > da7b6439c8d3803286b773a56c4b9a38776b5cd0beb8fd628b6007df235cf35c > da7fd412b79358f84b7928d2f6b7ccdaf165a21313608e16edd317a5355ba250 Since we are only concerned with the leading binary bits close to the neighborhood depth, for the rest of this example we will abbreviate the addresses to the first four prefixed hexadecimal digits only. Below are listed the hex prefixes and their binary representation, with the first ten leading bits underlined: | Hex prefix | Binary Bits | |------------|-----------------| | da4c | 1101101001001100| | da5d | 1101101001011101| | da76 | 1101101001110110| | da7a | 1101101001111010| | da7b | 1101101001111011| | da7f | 1101101001111111| ### Area of Responsibility Storer nodes are responsible for storing chunks with addresses whose leading bits match their own up to the storage depth. Here are two example chunks which fall within our example neighborhood: > Chunk A address: `da49a42926015cd1e2bc552147c567b1ca13e8d4302c9e6026e79a24de328b65` > Chunk B address: `da696a3dfb0f7f952872eb33e0e2a1435c61f111ff361e64203b5348cc06dc8a` As the address of the chunk shown above shares the same ten leading binary bits as the nodes in our example neighborhood, it falls into that neighborhood's [area of responsibility](./../../references/glossary.md#2-area-of-responsibility-related-depths), and all the nodes in that neighborhood are required to store that chunk: > da49 --> 1101101001001001 > da69 --> 1101101001101001 *As with the example for nodes, we've abbreviated the chunk addresses to their leading four hexadecimal digits only and converted them to binary digits.* ### Neighborhood Doubling As more and more chunks are assigned to neighborhoods, the chunk reserves of the nodes in that neighborhood will begin to fill up. Once the nodes' reserves in a neighborhood become full and can no longer store additional chunks, that neighborhood will split, with each half of the neighborhood taking responsibility for half of the chunks. This event is referred to as a "doubling", as it results in double the number of neighborhoods. The split is done by increasing the storage depth by one, so that the number of shared leading bits is increased by one. This results in a binary splitting of the neighborhood and associated chunks into two new neighborhoods and respective groups of chunks. :::info Note that when chunks begin to expire and new chunks are not uploaded to Swarm, it is possible for node's reserves to empty out, once they fall below a certain threshold, a "halving" will occur in which the storage depth will be decreased by one and two neighborhoods will merge to make a new one so that they are responsible for a wider set of chunks. ::: Using our previous example neighborhood, during a doubling, the storage depth would increase from 10 to 11, and the neighborhood would be split based on the 11th leading bit. **neighborhood A:** | Hex prefix | Binary Bits | |------------|-----------------| | da4c | 1101101001001100| | da5d | 1101101001011101| **neighborhood B:** | Hex prefix | Binary Bits | |------------|-----------------| | da76 | 1101101001110110| | da7a | 1101101001111010| | da7b | 1101101001111011| | da7f | 1101101001111111| Each of our two example chunks will also be split amongst the two new neighborhoods based on their 11th leading bit: **neighborhood A:** | Hex prefix | Binary Bits | |------------|-----------------| | da4c | 1101101001001100| | da5d | 1101101001011101| |da49 (chunk)| 1101101001001001| **neighborhood B:** | Hex prefix | Binary Bits | |------------|-----------------| | da76 | 1101101001110110| | da7a | 1101101001111010| | da7b | 1101101001111011| | da7f | 1101101001111111| |da69 (chunk)| 1101101001101001| #### Doubling Implications for Node Operators One of the implications of doubling for node operators is that the reward chances for a node depends in part on how many other nodes are in its neighborhood. If it is in a neighborhood with fewer nodes, its chances of winning rewards are greater. Therefore node operators should make certain to place their nodes into less populated neighborhoods, and also should look ahead to neighborhoods at the next depth after a doubling. For more details about how to adjust node placement, see [here](./../../bee/installation/set-target-neighborhood.md). --- ## Access Control The Access Control Trie (ACT) implements the operation of encryption at the chunk level, with the presence of a decryption/encryption key being the only distinction between accessing private and public data. :::info This article describes the high level concepts and functionalities of ACT. If you're ready to try it out for yourself, please refer to this [hands on usage guide with specific details](./../develop/access-control.md). ::: In decentralized public data storage systems like Swarm, data is distributed across multiple nodes. Ensuring confidentiality, integrity, and availability becomes paramount. The Access Control Trie (ACT) addresses these challenges by managing access control information for Swarm nodes. ## Key Concepts From the perspective of access controlled content, we can identify two main roles: | Role | Rights & responsibilities | |------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------| | **Content Publisher** | Publishers upload data and grant access to viewers based on their wallets’ public keys.They can also revoke access from specific viewers. | | **Grantee (Content Viewer)** | Grantees can access the content version allowed by the publisher.However, they may be blocked from accessing new versions of the content. | The control is defined by a process to obtain the full (decrypted) reference to the protected content uploaded by the publisher, which makes granted access possible. For the management of access by multiple grantees (viewers), an additional layer is introduced to derive the access key from their specific session key. This data structure, the lookup table for ACT, is implemented as key-value store in a Swarm manifest format. The publisher is able to add and remove grantees from this ACT. ### Session For each grantee, their public key is used as the session key. Using Diffie-Hellman key derivation, two additional keys will be derived from the session key: a lookup key and an access key decryption key (used for symmetric encryption of the access key). This means each grantee will have the content's access key specifically encrypted for them, and only they will be able to decrypt this, thus gain access to the content. ### ACT lookup table The ACT lookup table is a key-value store implemented over a Swarm manifest. It holds lookup keys and encrypted access keys prepared for the grantees when they are added to the ACT (granting access to the content). ### History The history of the ACT is maintained as well. This allows to retrieve a historical version of the ACT based on the timestamp attached to it. This also ensures that grantees will be able to retrieve the content version they were granted access to (using the relevant timestamp), even if their access to newer versions were revoked. ### Encryption It is important to emphasise that all elements of the process will undergo encryption. Including the grantee list itself, which is encrypted using the publisher’s own lookup key, as well as the grantee list’s content reference. This ensures that the security of the process and the data is always maintained. --- ## Bandwidth Incentives (SWAP) The Swarm Accounting Protocol (SWAP) is a protocol used to manage the exchange of bandwidth resources between nodes. SWAP ensures that node operators collaborate in routing messages and data while protecting the network against frivolous use of bandwidth. The protocol combines off-chain peer-to-peer based accounting with on-chain settlement through the chequebook contract. :::info[Key facts] - **What it is**: SWAP (Swarm Accounting Protocol) manages the exchange of bandwidth between nodes. - **Accounting**: each pair of peers tracks their relative bandwidth usage off-chain. - **Settlement**: when one node's debt crosses a threshold, it either issues an xBZZ "cheque" (settled on-chain via the chequebook contract) or keeps serving bandwidth in kind until the debt is paid off. - **Thresholds & freeloaders**: each node sets its own debt threshold; nodes that do not pay risk being blacklisted. ::: As nodes relay requests and responses, they keep track of their bandwidth usage with each of their peers. Peers engage in a service-for-service exchange, where they provide resources to each other based on their relative usage. Once a node's relative debt with one of their peers crosses a certain threshold, the party in debt can either send a xBZZ payment in the form of a "cheque" (an off chain commitment to pay their debt), or can continue to provide bandwidth services in kind until their debt is paid off. Each node can set their own threshold for the level of relative debt they accept. Freeloader nodes which do not pay their debts are at risk of being blacklisted by other nodes. ## Chequebook Contract The [chequebook contract](https://github.com/ethersphere/swap-swear-and-swindle/blob/master/contracts/ERC20SimpleSwap.sol) is a smart contract used in the SWAP protocol to manage cheques that are sent between nodes on the network. It acts as a wallet which nodes can fund with xBZZ which can be used to issue payments when presented with a valid cheque. The contract is also responsible for ensuring that cheques are valid and are cashed out correctly. When a node sends a cheque to one of its peers, it includes a signed message that specifies the amount of xBZZ tokens being transferred and the recipient's address. The chequebook contract receives this message and verifies that it is valid by checking the signature and ensuring that the sender has enough funds to cover the transfer. If the cheque is valid, the contract updates the balances of both nodes accordingly. The recipient can then cash out their xBZZ tokens by sending a transaction to the blockchain that invokes a function in the chequebook contract. This function transfers the specified amount of xBZZ tokens from the sender's account to the recipient's account. ## Opportunistic Caching When a node serves a chunk, the chunk is saved in the nodes' cache. Popular chunks which are frequently requested are kept in the cache so that they can be served again without the need to re-download the chunks from the network. This allows nodes to maximise their earnings by retaining popular chunks. This mechanism also contributes to Swarm's scalability, as popular chunks are always readily available for download as a result of opportunistic caching. To learn in more detail about how bandwidth incentives work, refer to sections 3.1 and 3.2 from [The Book of Swarm](https://papers.ethswarm.org/p/book-of-swarm/). --- ## Incentives Overview A key challenge in decentralized data networks is incentivizing users to store and transmit data. Swarm addresses this with two incentive mechanisms: **storage incentives**, which reward nodes for storing data, and **bandwidth incentives**, which reward nodes for relaying data. Together, these mechanisms establish a self-sustaining economic system where nodes are compensated for contributing resources honestly. Swarm's storage incentives are detailed in the [Future Proof Storage](https://www.ethswarm.org/swarm-storage-incentives.pdf) paper and [The Book of Swarm](https://papers.ethswarm.org/p/book-of-swarm/). ## Storage Incentives Storage incentives reward node operators for providing disk space and reliably storing data. The system is governed by three interconnected smart contracts: - **Postage Stamp Contract** – Handles payments for uploading data by way of purchasing "postage stamp batches". - **Redistribution Contract** – Distributes payments for postage stamps to nodes that store data. - **Price Oracle Contract** – Uses network redundancy data to determine postage stamp prices. If you want to dig into the code, check out the [incentives contracts repo](https://github.com/ethersphere/storage-incentives) You can find the on-chain address for each contract within the docs [here](./../../references/smart-contracts.mdx#storage-incentives-contracts), however since the contracts there are updated manually, they may at times fall slightly behind the most recent changes. For the most up to date address for each storage incentives contract refer to the [storage incentives ABI repo](https://github.com/ethersphere/go-storage-incentives-abi/commits/master/abi/abi_mainnet.go), and you can also find past addresses of older versions of the incentives contracts by reviewing previous commits. ### Postage Stamps Postage stamps are required to upload data to Swarm, similar to how real-world postage stamps prepay for mail delivery. Instead of being purchased individually, they are bought in batches using xBZZ through the postage stamp smart contract. The xBZZ used to buy postage stamps is later redistributed as storage incentives. The **price oracle contract** adjusts postage stamp pricing based on network redundancy to ensure a sustainable level of storage. You can find more details about postage stamps [here](./postage-stamps.md). ### Redistribution Game The redistribution game determines how xBZZ from postage stamp purchases is distributed among full staking nodes that store data. The system is designed so that **honestly storing assigned data** is the most profitable strategy. Rules for this process are encoded in the [redistribution smart contract](https://github.com/ethersphere/storage-incentives). Additionally, the game generates a **utilization signal**, which the price oracle uses to regulate postage stamp prices. Read more [here](./redistribution-game.md). ### Price Oracle The price oracle contract dynamically adjusts postage stamp prices based on network utilization data from the redistribution contract. This mechanism ensures optimal redundancy by increasing or decreasing the price of storage as needed. [Read more](./price-oracle.md). ## Bandwidth Incentives Nodes in Swarm not only store data but relay data across the network. **Bandwidth incentives** compensate nodes for these services. The **Swarm Accounting Protocol (SWAP)** facilitates bandwidth payments between nodes, which can be settled either **in-kind** (data exchange) or via **cheques** processed through a **chequebook contract** on Gnosis Chain. Only full nodes can participate in SWAP. Read more [here](./bandwidth-incentives.md). --- ## Postage Stamps Postage stamps are used to pay for storing data on Swarm. They are purchased in batches, granting a prepaid right to store data on Swarm, similar to how real-world postage stamps pay for mail delivery. When a node uploads data to Swarm, it 'attaches' postage stamps to each [chunk](./../DISC/DISC.mdx) of data. The value assigned to a stamp indicates how much it is worth to persist the associated data on Swarm, which nodes use to prioritize which chunks to remove from their reserve first. The value of a postage stamp decreases over time as if storage rent was regularly deducted from the batch balance. A stamp expires when its batch runs out of balance. Chunks with expired stamps cannot be used as proof in the redistribution game, meaning storer nodes will no longer receive rewards for storing them and can safely remove them from their reserves. Postage stamp prices are dynamically set based on a utilization signal supplied by the price oracle smart contract. Prices will automatically increase or decrease according to the level of utilization. ## Batch Buckets Postage stamps are issued in batches with a certain number of storage slots partitioned into $$2^{bucketDepth}$$ equally sized address space buckets (bucket depth has a fixed value of 16). Each bucket is responsible for storing chunks that fall within a certain range of the address space. When uploaded, files are split into 4kb chunks, each chunk is assigned a unique address, and each chunk is then assigned to the bucket in which its address falls. ### Bucket Size Bucket depth determines how the address space is partitioned, with each bucket storing chunks that share a common address prefix. Each bucket contains a fixed number of slots, each capable of storing a stamped chunk. Once all slots in any single bucket are filled, the entire postage batch becomes fully utilized, preventing further uploads. Together with `batch depth`, `bucket depth` determines how many chunks are allowed in each bucket. The number of chunks allowed in each bucket is calculated like so: $$ 2^{(batchDepth - bucketDepth)} $$ So with a batch depth of 24 and a bucket depth of 16: $$ 2^{(24 - 16)} = 2^{8} = 256 \text{ chunks/bucket} $$ :::info Note that due how buckets fill as described above, a batch can become fully utilised before its theoretical maximum volume has been reached. See [batch utilisation section below](./postage-stamps.md#batch-utilisation) for more information. ::: ## Batch Depth and Batch Amount Each batch of stamps has two key parameters, `batch depth` and `amount`, which are recorded on Gnosis Chain at issuance. Note that these "depths" do not refer to the depth terms used to describe topology which are outlined [here in the glossary](./../../references/glossary.md#depth-types). ### Batch Depth :::caution The minimum value for `depth` is 17, however higher depths are recommended for most use cases due to the [mechanics of stamp batch utilisation](#batch-utilisation). See [the depths utilisation table](#effective-utilisation-tables) to help decide which depth is best for your use case. ::: `Batch depth` determines how much data can be stored by a batch. The number of chunks which can be stored (stamped) by a batch is equal to $$2^{batchDepth}$$. For a batch with a `batch depth` of 24, a maximum of $$2^{24} = 16,777,216$$ chunks can be stamped. Since we know that one chunk can store 4 kb of data, we can calculate the theoretical maximum amount of data which can be stored by a batch from the `batch depth`. $$ \text{Theoretical maximum batch volume} = 2^{batchDepth} \times \text{4 kb} $$ However, due to the way postage stamp batches are utilised, batches will become fully utilised before stamping the theoretical maximum number of chunks. Therefore when deciding which batch depth to use, it is important to consider the effective amount of data that can be stored by a batch, and not the theoretical maximum. The effective rate of utilisation increases along with the batch depth. See [section on stamp batch utilisation below](./postage-stamps.md#batch-utilisation) for more information. ### Batch Amount (& Batch Cost) The `amount` parameter is the quantity of xBZZ in PLUR $$(1 \times 10^{16}PLUR = 1 \text{ xBZZ})$$ that is assigned per chunk in the batch. The total number of xBZZ that will be paid for the batch is calculated from this figure and the `batch depth` like so: $$2^{batchDepth} \times {amount}$$ The paid xBZZ forms the `balance` of the batch. This `balance` is then slowly depleted as time ticks on and blocks are mined on Gnosis Chain. For example, with a `batch depth` of 24 and an `amount` of 1000000000 PLUR: $$ 2^{24} \times 1000000000 = 16777216000000000 \text{ PLUR} = 1.6777216 \text{ xBZZ} $$ ### Calculating *amount* needed for desired TTL To calculate the required `amount`, divide the current postage price by the Gnosis block time (5 sec) and multiply by the desired storage duration in seconds. For the example below we assume a stamp price of 24000 PLUR / chunk / block: :::info The postage stamp price is dynamically determined according to a network utilisation signal. You can view the current storage price at [Swarmscan.io](https://swarmscan.io/). ::: $$ (\text{stamp price} \div \text{block time in seconds}) \times \text{storage time in seconds} $$ There are 1036800 seconds in 12 days, so the `amount` value required to store for 12 days can be calculated: $$ (\text{24000} \div \text{5}) \times \text{1036800} = 4976640000 $$ So we can use 4976640000 as our `amount` value in order for our postage batch to store data for 12 days. ## Batch Utilisation There are two types of postage stamp batches: immutable and mutable. Immutable batches permanently store data, while mutable batches allow overwriting older data as new chunks are added. ### Immutable Batches Utilisation of an immutable batch is computed using a hash map of size $$2^{bucketDepth}$$ which is $$2^{16}$$ for all batches, so 65536 total entries. For the keys of the key-value pairs of the hash map, the keys are 16 digit binary numbers from 0 to 65535, and the value is a counter. ![](/img/batches_01.png) As chunks are uploaded to Swarm, each chunk is assigned to a bucket based the first 16 binary digits of the [chunk's hash](./../DISC/DISC.mdx#chunks). The chunk will be assigned to whichever bucket's key matches the first 16 bits of its hash, and that bucket's counter will be incremented by 1. The batch is deemed "full" when ANY of these counters reach a certain max value. The max value is computed from the batch depth as such: $$2^{(batchDepth-bucketDepth)}$$. For example with batch depth of 24, the max value is $$2^{(24-16)}$$ or 256. A bucket can be thought of as have a number of "slots" equal to this maximum value, and every time the bucket's counter is incremented, one of its slots gets filled. In the diagram below, the batch depth is 18, so there are $$2^{(18-16)}$$ or 4 slots for each bucket. The utilisation of a batch is simply the highest number of filled slots out of all 65536 entries or "buckets". In this batch, none of the slots in any of the buckets have yet been filled with 4 chunks, so the batch is not yet fully utilised. The most filled slots out of all buckets is 2, so the stamp batch's utilisation is 2 out of 4. ![](/img/batches_02.png) As more chunks get uploaded and stamped, the bucket slots will begin to fill. As soon as the slots for any SINGLE bucket get filled, the entire batch is considered 100% utilised and can no longer be used to upload additional chunks. ![](/img/batches_03.png) ### Mutable Batches Mutable batches use the same hash map structure as immutable batches, however its utilisation works very differently. In contrast with immutable batches, mutable batches are never considered fully utilised. Rather, at the point where an immutable batch would be considered fully utilised, a mutable batch can continue to stamp chunks. However, if any chunk's address lands in a bucket whose slots are already filled, rather than the batch becoming fully utilised, that bucket's counter gets reset, and the new chunk will replace the oldest chunk in that bucket. ![](/img/batches_04.png) Therefore rather than speaking of the number of slots as determining the utilisation of a batch as with immutable batches, we can think of the slots as defining a limit to the amount of data which can be uploaded before old data starts to get overwritten. ### Which Type of Batch to Use | | Immutable batch | Mutable batch | |---|---|---| | Data retention | Uploaded data won't be overwritten by later uploads to the same batch | Older data may be overwritten once capacity is reached | | When capacity is full | Additional uploads are not accepted | Keeps accepting uploads; overwrites the oldest chunks | | Default? | Yes (`immutable` unset) | No (set the `immutable` header to `false`) | | Best for | Long-term or never-overwritten data (archives, legal documents, photos) | Frequently updated data (blogs, websites, messaging) | Immutable batches are suitable for long term storage of data or for data which otherwise does not need to be changed and should never be overwritten, such as records archival, legal documents, family photos, etc. Mutable batches are great for data which needs to be frequently updated and does not require a guarantee of immutability. For example, a blog, personal or company websites, ephemeral messaging app, etc. The default batch type when unspecified is immutable. This can be modified through the Bee api by setting the `immutable` header with the [`\stamps POST` endpoint](https://docs.ethswarm.org/api/#tag/Transaction/paths/~1transactions~1%7BtxHash%7D/post) to `false`. ### Re-uploading There are several nuances to how the re-uploading of previously uploaded data to Swarm affect stamp batch utilisation. For single chunks, the behaviour is relatively straightforward, however with files that must get split into multiple chunks, the behaviour is less straightforward. #### Single chunks When a chunk which has previously been uploaded to Swarm is re-uploaded from the same node while the initial postage batch it was stamped by is still valid, no additional stamp will be utilised from the batch. However if the chunk comes from a different node than the original node, then a stamp WILL be utilised, and as long as at least one of the batches the chunk was stamped by is still valid, the chunk will be retained by storer nodes in its neighborhood. #### Files When an identical file is re-uploaded then the stamp utilisation behaviour will be the same as with single chunks described in the section above. However, if part of the file has been modified and then re-uploaded, stamp utilisation behaviour will be different. This is due to how the chunking process works when a file is uploaded to Swarm. When uploaded to Swarm, files are split into 4kb sized chunks (2^12 bytes), and each chunk is assigned an address which is based on the content of the chunk. If even a single bit within the chunk is modified, then the address of the chunk will also be modified. When a file which was previously uploaded with a single bit flipped is again split into chunks by a node before being uploaded to Swarm, then only the chunk with the flipped bit will have an updated address and require the utilisation of another stamp. The content of all the other chunks will remain the same, and therefore will not require new stamps to be utilised. However, if rather than flipping a single bit we add some data to our file, this could cause changes in the content of every chunk of the file, meaning that every single chunk must be re-stamped. We can use a simplified example of why this is the case to more easily understand the stamp utilisation behaviour. Let us substitute a message containing letters of the alphabet rather than binary data. Our initial message consists of 16 letters: > abcdefghijklmnop When initially uploaded, it will be split into four chunks of four letters each: > abcdefghijklmnop => abcd | efgh | ijkl | mnop Let us look at what happens when a single letter is changed (here we change a to z): > abcdefghijklmnop => zbcd | efgh | ijkl | mnop In this case, only the first chunk is affected, all the other chunks retain the same content. > Now let is examine the case where a new letter is added rather than simply modifying an already existing one. Here we add the number 1 at the start of the message: > 1abcdefghijklmnop => 1abc | defg | hijk | lmno | p As you can see, by adding a single new letter at the start of the message, all the letters are shifted to the right by a single position, which a has caused EVERY chunk in the message to be modified rather than just a single chunk. #### Affect on Batch Utilisation The implications of this behaviour are that even a small change to the data of a file may cause every single chunk from the file to be changed, meaning that new stamps must be utilised for every chunk from that file. In practice, this could lead to high costs in data which is frequently changed, since for even a small change, every chunk from the file must be re-stamped. ### Implications for Swarm Users Because of how buckets fill during batch utilisation, batches are often fully utilised before reaching their theoretical maximum storage amount. However as the batch depth increases, the chance of a postage batch becoming fully utilised early decreases. At batch depth 24 (unencrypted, no erasure coding), there is a 0.1% chance that a batch will be fully utilised/start replacing old chunks before reaching 68.48% of its theoretical maximum. Let's look at an example to make it clearer. Using the method of calculating the theoretical maximum storage amount [outlined above](./postage-stamps.md#batch-depth), we can see that for a batch depth of 24, the theoretical maximum amount which can be stored is 68.72 gb: $$ 2^{24+12} = \text{68,719,476,736 bytes} = \text{68.72 gb} $$ Therefore we should use 68.48% the effective rate of usage for the stamp batch: $$ \text{68.72 gb} \times{0.6848} = \text{47.06 gb } $$ Note that the effective volume also depends on the encryption and erasure coding settings used. The example above assumes unencrypted data with no erasure coding. See the [effective utilisation tables below](#effective-utilisation-tables) for the full set of effective volumes. ## Effective Utilisation Tables When a user buys a batch of stamps they may make the naive assumption that they will be able to upload data equal to the sum total size of the maximum capacity of the batch. However, in practice this assumption is incorrect, so it is essential that Swarm users understand the relationship between batch depth and the theoretical and effective volumes of a batch. Columns: * **Theoretical Volume:** The theoretical maximum volume which can be reached if the batch is completely utilized. * **Effective Volume:** The actual volume which a batch can be expected to store with a failure rate of less than or equal to 0.1% (1 in 1000). * **Batch Depth:** The batch depth value. :::info The title of each table below states whether it is for encrypted or unencrypted uploads along with the erasure coding level. [Erasure coding](./../DISC/erasure-coding.md) on Swarm has five named levels: 1. NONE 2. MEDIUM 3. STRONG 4. INSANE 5. PARANOID ::: ### Unencrypted - NONE | Theoretical Volume | Effective Volume | Batch Depth | | ------------- | ------------- | ------------- | | 536.87 MB | 44.70 kB | 17 | | 1.07 GB | 6.66 MB | 18 | | 2.15 GB | 112.06 MB | 19 | | 4.29 GB | 687.62 MB | 20 | | 8.59 GB | 2.60 GB | 21 | | 17.18 GB | 7.73 GB | 22 | | 34.36 GB | 19.94 GB | 23 | | 68.72 GB | 47.06 GB | 24 | | 137.44 GB | 105.51 GB | 25 | | 274.88 GB | 227.98 GB | 26 | | 549.76 GB | 476.68 GB | 27 | | 1.10 TB | 993.65 GB | 28 | | 2.20 TB | 2.04 TB | 29 | | 4.40 TB | 4.17 TB | 30 | | 8.80 TB | 8.45 TB | 31 | | 17.59 TB | 17.07 TB | 32 | | 35.18 TB | 34.36 TB | 33 | | 70.37 TB | 69.04 TB | 34 | | 140.74 TB | 138.54 TB | 35 | | 281.47 TB | 277.72 TB | 36 | | 562.95 TB | 556.35 TB | 37 | | 1.13 PB | 1.11 PB | 38 | | 2.25 PB | 2.23 PB | 39 | | 4.50 PB | 4.46 PB | 40 | | 9.01 PB | 8.93 PB | 41 | ### Unencrypted - MEDIUM | Theoretical Volume | Effective Volume | Batch Depth | | ------------- | ------------- | ------------- | | 536.87 MB | 41.56 kB | 17 | | 1.07 GB | 6.19 MB | 18 | | 2.15 GB | 104.18 MB | 19 | | 4.29 GB | 639.27 MB | 20 | | 8.59 GB | 2.41 GB | 21 | | 17.18 GB | 7.18 GB | 22 | | 34.36 GB | 18.54 GB | 23 | | 68.72 GB | 43.75 GB | 24 | | 137.44 GB | 98.09 GB | 25 | | 274.88 GB | 211.95 GB | 26 | | 549.76 GB | 443.16 GB | 27 | | 1.10 TB | 923.78 GB | 28 | | 2.20 TB | 1.90 TB | 29 | | 4.40 TB | 3.88 TB | 30 | | 8.80 TB | 7.86 TB | 31 | | 17.59 TB | 15.87 TB | 32 | | 35.18 TB | 31.94 TB | 33 | | 70.37 TB | 64.19 TB | 34 | | 140.74 TB | 128.80 TB | 35 | | 281.47 TB | 258.19 TB | 36 | | 562.95 TB | 517.23 TB | 37 | | 1.13 PB | 1.04 PB | 38 | | 2.25 PB | 2.07 PB | 39 | | 4.50 PB | 4.15 PB | 40 | | 9.01 PB | 8.30 PB | 41 | ### Unencrypted - STRONG | Theoretical Volume | Effective Volume | Batch Depth | | ------------- | ------------- | ------------- | | 536.87 MB | 37.37 kB | 17 | | 1.07 GB | 5.57 MB | 18 | | 2.15 GB | 93.68 MB | 19 | | 4.29 GB | 574.81 MB | 20 | | 8.59 GB | 2.17 GB | 21 | | 17.18 GB | 6.46 GB | 22 | | 34.36 GB | 16.67 GB | 23 | | 68.72 GB | 39.34 GB | 24 | | 137.44 GB | 88.20 GB | 25 | | 274.88 GB | 190.58 GB | 26 | | 549.76 GB | 398.47 GB | 27 | | 1.10 TB | 830.63 GB | 28 | | 2.20 TB | 1.71 TB | 29 | | 4.40 TB | 3.49 TB | 30 | | 8.80 TB | 7.07 TB | 31 | | 17.59 TB | 14.27 TB | 32 | | 35.18 TB | 28.72 TB | 33 | | 70.37 TB | 57.71 TB | 34 | | 140.74 TB | 115.81 TB | 35 | | 281.47 TB | 232.16 TB | 36 | | 562.95 TB | 465.07 TB | 37 | | 1.13 PB | 931.23 TB | 38 | | 2.25 PB | 1.86 PB | 39 | | 4.50 PB | 3.73 PB | 40 | | 9.01 PB | 7.46 PB | 41 | ### Unencrypted - INSANE | Theoretical Volume | Effective Volume | Batch Depth | | ------------- | ------------- | ------------- | | 536.87 MB | 33.88 kB | 17 | | 1.07 GB | 5.05 MB | 18 | | 2.15 GB | 84.92 MB | 19 | | 4.29 GB | 521.09 MB | 20 | | 8.59 GB | 1.97 GB | 21 | | 17.18 GB | 5.86 GB | 22 | | 34.36 GB | 15.11 GB | 23 | | 68.72 GB | 35.66 GB | 24 | | 137.44 GB | 79.96 GB | 25 | | 274.88 GB | 172.77 GB | 26 | | 549.76 GB | 361.23 GB | 27 | | 1.10 TB | 753.00 GB | 28 | | 2.20 TB | 1.55 TB | 29 | | 4.40 TB | 3.16 TB | 30 | | 8.80 TB | 6.41 TB | 31 | | 17.59 TB | 12.93 TB | 32 | | 35.18 TB | 26.04 TB | 33 | | 70.37 TB | 52.32 TB | 34 | | 140.74 TB | 104.99 TB | 35 | | 281.47 TB | 210.46 TB | 36 | | 562.95 TB | 421.61 TB | 37 | | 1.13 PB | 844.20 TB | 38 | | 2.25 PB | 1.69 PB | 39 | | 4.50 PB | 3.38 PB | 40 | | 9.01 PB | 6.77 PB | 41 | ### Unencrypted - PARANOID | Theoretical Volume | Effective Volume | Batch Depth | | ------------- | ------------- | ------------- | | 536.87 MB | 13.27 kB | 17 | | 1.07 GB | 1.98 MB | 18 | | 2.15 GB | 33.27 MB | 19 | | 4.29 GB | 204.14 MB | 20 | | 8.59 GB | 771.13 MB | 21 | | 17.18 GB | 2.29 GB | 22 | | 34.36 GB | 5.92 GB | 23 | | 68.72 GB | 13.97 GB | 24 | | 137.44 GB | 31.32 GB | 25 | | 274.88 GB | 67.68 GB | 26 | | 549.76 GB | 141.51 GB | 27 | | 1.10 TB | 294.99 GB | 28 | | 2.20 TB | 606.90 GB | 29 | | 4.40 TB | 1.24 TB | 30 | | 8.80 TB | 2.51 TB | 31 | | 17.59 TB | 5.07 TB | 32 | | 35.18 TB | 10.20 TB | 33 | | 70.37 TB | 20.50 TB | 34 | | 140.74 TB | 41.13 TB | 35 | | 281.47 TB | 82.45 TB | 36 | | 562.95 TB | 165.17 TB | 37 | | 1.13 PB | 330.72 TB | 38 | | 2.25 PB | 661.97 TB | 39 | | 4.50 PB | 1.32 PB | 40 | | 9.01 PB | 2.65 PB | 41 | ### Encrypted - NONE | Theoretical Volume | Effective Volume | Batch Depth | | ------------- | ------------- | ------------- | | 536.87 MB | 44.35 kB | 17 | | 1.07 GB | 6.61 MB | 18 | | 2.15 GB | 111.18 MB | 19 | | 4.29 GB | 682.21 MB | 20 | | 8.59 GB | 2.58 GB | 21 | | 17.18 GB | 7.67 GB | 22 | | 34.36 GB | 19.78 GB | 23 | | 68.72 GB | 46.69 GB | 24 | | 137.44 GB | 104.68 GB | 25 | | 274.88 GB | 226.19 GB | 26 | | 549.76 GB | 472.93 GB | 27 | | 1.10 TB | 985.83 GB | 28 | | 2.20 TB | 2.03 TB | 29 | | 4.40 TB | 4.14 TB | 30 | | 8.80 TB | 8.39 TB | 31 | | 17.59 TB | 16.93 TB | 32 | | 35.18 TB | 34.09 TB | 33 | | 70.37 TB | 68.50 TB | 34 | | 140.74 TB | 137.45 TB | 35 | | 281.47 TB | 275.53 TB | 36 | | 562.95 TB | 551.97 TB | 37 | | 1.13 PB | 1.11 PB | 38 | | 2.25 PB | 2.21 PB | 39 | | 4.50 PB | 4.43 PB | 40 | | 9.01 PB | 8.86 PB | 41 | ### Encrypted - MEDIUM | Theoretical Volume | Effective Volume | Batch Depth | | ------------- | ------------- | ------------- | | 536.87 MB | 40.89 kB | 17 | | 1.07 GB | 6.09 MB | 18 | | 2.15 GB | 102.49 MB | 19 | | 4.29 GB | 628.91 MB | 20 | | 8.59 GB | 2.38 GB | 21 | | 17.18 GB | 7.07 GB | 22 | | 34.36 GB | 18.24 GB | 23 | | 68.72 GB | 43.04 GB | 24 | | 137.44 GB | 96.50 GB | 25 | | 274.88 GB | 208.52 GB | 26 | | 549.76 GB | 435.98 GB | 27 | | 1.10 TB | 908.81 GB | 28 | | 2.20 TB | 1.87 TB | 29 | | 4.40 TB | 3.81 TB | 30 | | 8.80 TB | 7.73 TB | 31 | | 17.59 TB | 15.61 TB | 32 | | 35.18 TB | 31.43 TB | 33 | | 70.37 TB | 63.15 TB | 34 | | 140.74 TB | 126.71 TB | 35 | | 281.47 TB | 254.01 TB | 36 | | 562.95 TB | 508.85 TB | 37 | | 1.13 PB | 1.02 PB | 38 | | 2.25 PB | 2.04 PB | 39 | | 4.50 PB | 4.08 PB | 40 | | 9.01 PB | 8.17 PB | 41 | ### Encrypted - STRONG | Theoretical Volume | Effective Volume | Batch Depth | | ------------- | ------------- | ------------- | | 536.87 MB | 36.73 kB | 17 | | 1.07 GB | 5.47 MB | 18 | | 2.15 GB | 92.07 MB | 19 | | 4.29 GB | 564.95 MB | 20 | | 8.59 GB | 2.13 GB | 21 | | 17.18 GB | 6.35 GB | 22 | | 34.36 GB | 16.38 GB | 23 | | 68.72 GB | 38.66 GB | 24 | | 137.44 GB | 86.69 GB | 25 | | 274.88 GB | 187.31 GB | 26 | | 549.76 GB | 391.64 GB | 27 | | 1.10 TB | 816.39 GB | 28 | | 2.20 TB | 1.68 TB | 29 | | 4.40 TB | 3.43 TB | 30 | | 8.80 TB | 6.94 TB | 31 | | 17.59 TB | 14.02 TB | 32 | | 35.18 TB | 28.23 TB | 33 | | 70.37 TB | 56.72 TB | 34 | | 140.74 TB | 113.82 TB | 35 | | 281.47 TB | 228.18 TB | 36 | | 562.95 TB | 457.10 TB | 37 | | 1.13 PB | 915.26 TB | 38 | | 2.25 PB | 1.83 PB | 39 | | 4.50 PB | 3.67 PB | 40 | | 9.01 PB | 7.34 PB | 41 | ### Encrypted - INSANE | Theoretical Volume | Effective Volume | Batch Depth | | ------------- | ------------- | ------------- | | 536.87 MB | 33.26 kB | 17 | | 1.07 GB | 4.96 MB | 18 | | 2.15 GB | 83.38 MB | 19 | | 4.29 GB | 511.65 MB | 20 | | 8.59 GB | 1.93 GB | 21 | | 17.18 GB | 5.75 GB | 22 | | 34.36 GB | 14.84 GB | 23 | | 68.72 GB | 35.02 GB | 24 | | 137.44 GB | 78.51 GB | 25 | | 274.88 GB | 169.64 GB | 26 | | 549.76 GB | 354.69 GB | 27 | | 1.10 TB | 739.37 GB | 28 | | 2.20 TB | 1.52 TB | 29 | | 4.40 TB | 3.10 TB | 30 | | 8.80 TB | 6.29 TB | 31 | | 17.59 TB | 12.70 TB | 32 | | 35.18 TB | 25.57 TB | 33 | | 70.37 TB | 51.37 TB | 34 | | 140.74 TB | 103.08 TB | 35 | | 281.47 TB | 206.65 TB | 36 | | 562.95 TB | 413.98 TB | 37 | | 1.13 PB | 828.91 TB | 38 | | 2.25 PB | 1.66 PB | 39 | | 4.50 PB | 3.32 PB | 40 | | 9.01 PB | 6.64 PB | 41 | ### Encrypted - PARANOID | Theoretical Volume | Effective Volume | Batch Depth | | ------------- | ------------- | ------------- | | 536.87 MB | 13.17 kB | 17 | | 1.07 GB | 1.96 MB | 18 | | 2.15 GB | 33.01 MB | 19 | | 4.29 GB | 202.53 MB | 20 | | 8.59 GB | 765.05 MB | 21 | | 17.18 GB | 2.28 GB | 22 | | 34.36 GB | 5.87 GB | 23 | | 68.72 GB | 13.86 GB | 24 | | 137.44 GB | 31.08 GB | 25 | | 274.88 GB | 67.15 GB | 26 | | 549.76 GB | 140.40 GB | 27 | | 1.10 TB | 292.67 GB | 28 | | 2.20 TB | 602.12 GB | 29 | | 4.40 TB | 1.23 TB | 30 | | 8.80 TB | 2.49 TB | 31 | | 17.59 TB | 5.03 TB | 32 | | 35.18 TB | 10.12 TB | 33 | | 70.37 TB | 20.34 TB | 34 | | 140.74 TB | 40.80 TB | 35 | | 281.47 TB | 81.80 TB | 36 | | 562.95 TB | 163.87 TB | 37 | | 1.13 PB | 328.11 TB | 38 | | 2.25 PB | 656.76 TB | 39 | | 4.50 PB | 1.31 PB | 40 | | 9.01 PB | 2.63 PB | 41 | --- ## Price Oracle ## How does the price oracle set stamp prices? {#stamp-prices} The price oracle targets a fourfold (4×) data-redundancy level as a safe minimum, and moves the stamp price to hold it there: when redundancy falls below 4 it raises the price, and when redundancy rises above 4 it lowers it — a negative-feedback loop that pulls redundancy back toward 4. The job of the [oracle contract](https://github.com/ethersphere/storage-incentives/blob/master/src/PriceOracle.sol) is to set the price of postage stamps. The oracle contract uses data from the [redistribution contract](https://github.com/ethersphere/storage-incentives/blob/master/src/Redistribution.sol) in order to set the appropriate price for postage stamps through the [postage stamp contract](https://github.com/ethersphere/storage-incentives/blob/master/src/PostageStamp.sol). The data from the redistribution contract is used to calculate a "utilisation signal". This signal is an indicator of how much the Swarm network’s data storage capacity is being utilized. Specifically, the signal is a measure of data redundancy on the network. Redundancy is a measure of how many copies of each piece of data can be stored by the network. The protocol targets a fourfold level of data redundancy as a safe minimum. ## How does the price adjust to network demand? {#network-demand} The oracle runs a negative-feedback loop that keeps redundancy near the 4× target: ```mermaid flowchart TD T([Target: 4x data redundancy]) T -- redundancy falls below 4 --> U[Oracle raises the stamp price] U --> V[Fewer stamps bought] V -- redundancy rises --> T T -- redundancy rises above 4 --> X[Oracle lowers the stamp price] X --> Y[More stamps bought] Y -- redundancy falls --> T ``` For example, if there is an increase in postage stamps being purchased while the number of nodes remains constant, the data redundancy level will begin to fall as data storers’ available space begins to become reserved. If too many postage stamps are purchased without an equivalent increase in storage providers, the redundancy level may fall below four. In this case, the oracle will increase the price of postage stamps so that it becomes more expensive to store data on Swarm. The higher cost of storage will then lead to less postage stamps being purchased, and will push the redundancy level back up towards four. Conversely, if the amount of Stamps being purchased decreases while the number of storage provider nodes remains constant, the redundancy level will increase as there are fewer chunks of data to be distributed amongst the same number of nodes. In this case, the oracle will decrease the Postage Stamp price in order to promote more data storers to store their data on Swarm. The lower cost of storage will then lead to more Postage Stamps being purchased and push the redundancy level back down towards four. --- ## Redistribution Game The redistribution game distributes xBZZ collected from [postage stamp](./postage-stamps.md) purchases, rewarding nodes for providing storage. Redistribution rewards incentivize nodes to continue providing storage to the network. The game is designed so that the most profitable strategy for participants is to store their assigned data honestly. ## How does the redistribution game work? {#redistribution-game-details} Uploading data to Swarm requires purchasing postage stamp batches with xBZZ. The collected xBZZ is later redistributed as rewards to storage nodes. Every 152 Gnosis Chain blocks ***a single [neighborhood](./../DISC/neighborhoods.md)*** is selected to play the redistribution game. For each round of the game, one node from the selected neighborhood will have the chance to win a reward which is paid out from the accumulated xBZZ. The game has 3 phases, `commit`, `reveal`, and `claim`. In the `reveal` phase of a previous game, an "anchor" address is randomly generated and used to determine the neighborhood for the current round. In the `commit` phase, nodes issue an on-chain transaction including an encrypted hash of the data they are storing (the unencrypted hash is known as the "reserve commitment") along with the [depth](./../../references/glossary.md#2-area-of-responsibility-related-depths) for which they are reporting. This serves as an attestation of the data they are storing without revealing any other information. In the `reveal` phase, each node reveals the decryption key for their encrypted hashes thereby publishing the hash. The winner is chosen at random among the honest nodes, but it is weighted in proportion to each node's stake density. Stake density is calculated as so: $$ \text{stake density} = \text{stake(xBZZ)} \times {2}^\text{storage depth} $$ ## What penalties apply for dishonest nodes? {#penalties} During the `reveal` phase if a nodes' revealed hash does not match the honest nodes' hash, that node will be temporarily frozen and will not be able to participate in a number of upcoming rounds. Currently the freeze period is defined in the [redistribution smart contract](https://github.com/ethersphere/storage-incentives/blob/master/src/Redistribution.sol#L536C1-L536C100) as: $$ 152 \times 2^\text{storage radius} \text{ blocks (at 5s per block)} $$ So for example at a storage radius of 10: $$ 152 \times 2^{10} \text{ blocks (at 5s per block)} ≈ \text{ 9 days} $$ --- ## Introduction Swarm is a peer-to-peer network of Bee nodes that collectively provide censorship-resistant decentralised storage and communication services. Swarm's mission is to enable a self-sovereign global society and permissionless open markets by providing scalable decentralized storage infrastructure for Web3. Its incentive system is enforced through smart contracts on the Gnosis Chain blockchain and powered by the xBZZ token, making it economically self-sustaining. ## Bee Client Bee is a Swarm client implemented in Go and serves as the foundation of the Swarm network. Bee nodes form a private, decentralized, and self-sustaining network for permissionless publishing and data storage. You can learn more about how Bee clients work by reading about the [concepts and protocols](./what-is-swarm.mdx) which underpin the Swarm network. To get hands on experience working with Swarm, you can start by learning how to [install and operate a Bee node](./../bee/installation/getting-started.md). ## Swarm Foundation The [Swarm Foundation](https://www.ethswarm.org/foundation) is dedicated to advancing open-source technology for decentralized data storage and exchange. It fosters a sustainable, independent ecosystem by supporting the development of free and open-source software (FLOSS) and empowering a community built around crypto-economic incentives for processing, distributing, and storing data. Its mission is to champion digital freedom by promoting the Swarm network as the foundational layer of the fair data economy, while nurturing the community that sustains it. To achieve this, the foundation provides financial grants and other forms of support, evaluated on a case-by-case basis. --- ## PSS PSS, or Postal Service over Swarm, is a messaging protocol that enables users to send and receive messages over Swarm. It is an essential component of Swarm's infrastructure, providing secure, private, and efficient communication between nodes. :::info[Key facts] - **What it is**: PSS (Postal Service over Swarm) is Swarm's node-to-node messaging protocol. - **How delivery works**: a message is encrypted to the recipient and wrapped in a content-addressed chunk whose address falls in the recipient's neighborhood, so the push-sync protocol delivers it; only the recipient can decrypt it. - **Anonymous inbound**: senders can be previously unknown identities. - **Offline recipients**: mailboxing lets a message wait for a recipient who is not online. ::: ## Security PSS is designed to be secure by encrypting messages for the intended recipient and wrapping them with a topic in a content-addressed chunk. The chunk is crafted in such a way that its content address falls into the recipient's neighborhood, ensuring that delivery is naturally taken care of by the push-sync protocol. This ensures that messages are delivered only to the intended recipient's neighborhood and cannot be intercepted or read by unauthorized parties. While the chunk will be delivered to all members of the recipient's neighborhood, only the recipient will be able to decrypt the message using their private key. ## Privacy PSS also provides privacy by allowing users to receive messages from previously unknown identities. This makes it an ideal communication primitive for sending anonymous messages to public identities such as registrations or initial contact to start a thread by setting up secure communication. ## Efficiency Efficiency is another key feature of PSS. It uses direct node-to-node messaging in Swarm, which means that messages are delivered directly from one node to another without the need for intermediaries. This reduces latency and ensures that messages are delivered quickly and reliably. ## Mailboxing PSS also supports mailboxing, which allows users to deposit messages for download if the recipient is not online. This ensures that messages are not lost if the recipient is offline when they are sent. --- ## What is Swarm? Swarm is a peer-to-peer network of nodes which work together to provide decentralised storage and communication infrastructure. The complete vision of Swarm is described in detail in [The Book of Swarm](https://papers.ethswarm.org/p/book-of-swarm/) written by Swarm founder Viktor Tron, with further high level details described in the [whitepaper](https://papers.ethswarm.org/p/whitepaper/). More in depth low level implementation details can be found in the [Swarm Specification paper](https://papers.ethswarm.org/p/swarm-protocol-spec/). The latest research and technical papers from Swarm can be found on the ["Papers" section](https://papers.ethswarm.org/) of the Ethswarm homepage. Swarm can be divided into four main parts: 1. Underlay Network - A peer-to-peer network protocol to serve as underlay transport. Swarm's underlay network is built with [libp2p](https://libp2p.io/). 2. Overlay Network - An overlay network with protocols powering a distributed immutable store for chunks (fixed size data blocks). 3. Data Access Layer - A component providing high-level data access and defining APIs for base-layer features. 4. Application Layer - An application layer defining standards and outlining best practices for more elaborate use cases. Source: The Book of Swarm - Figure 1.1 - "Swarm’s Layered Design" Of these four main parts, parts 2 and 3 form the core of Swarm. ### 1. Underlay Network The first part of Swarm is a peer-to-peer network protocol that serves as the underlay transport. The underlay transport layer is responsible for establishing connections between nodes in the network and routing data between them. It provides a low-level communication channel that enables nodes to communicate with each other directly, without relying on any centralised infrastructure. Swarm is designed to be agnostic of the particular underlay transport used, as long as it satisfies certain requirements described in The Book of Swarm. As the [libp2p](https://libp2p.io/) library meets all these requirements it has been used to build the Swarm underlay network. ### 2. Overlay Network The second part of Swarm is an overlay network with protocols powering the [Distributed Immutable Store of Chunks (DISC)](./DISC/DISC.mdx). This layer is responsible for storing and retrieving data in a decentralised and secure manner. Swarm's overlay network is built on top of the underlay transport layer and uses [Kademlia](./DISC/kademlia.mdx) overlay routing to enable efficient and scalable communication between nodes. Kademlia is a distributed hash table (DHT) algorithm that allows nodes to locate each other in the network based on their unique identifier or hash. Swarm's DISC is an implementation of a Kademlia DHT optimized for storage. While the use of DHTs in distributed data storage protocols is common, for many implementations DHTs are used only for indexing file references. Swarm's DISC distinguishes itself from other implementations by instead breaking files into chunks and storing the chunks themselves directly within the DHT. Each chunk has a fixed size of 4kb and is distributed across the network using the DISC model. Each chunk has a unique address taken from the same namespace as the network node addresses that allows it to be located and retrieved by other nodes in the network. Swarm's distributed immutable storage provides several benefits, including data redundancy, tamper-proofing, and fault tolerance. Because data is stored across multiple nodes in the network, it can be retrieved even if some nodes fail or go offline. Built on top of the overlay network is also an [incentives layer](./incentives/overview.mdx) which guarantees that node operators which share their resources with the network are fairly rewarded for their services. ### 3. Data Access Layer The third part of Swarm is a component that provides high-level data access and defines APIs for base-layer features. This layer is responsible for providing an easy-to-use interface for developers to interact with Swarm's underlying storage and communication infrastructure. Swarm's high-level data access component provides [APIs that allow developers to perform various operations](/api/) on the network, including [uploading and downloading data](./../develop/upload-and-download.md) and searching for content. These APIs are designed to be simple and intuitive, making it easy for developers to build decentralised applications on top of Swarm. ### 4. Application Layer The fourth part of Swarm is an application layer that defines standards and outlines best practices for more elaborate use cases. This layer is responsible for providing guidance to developers on [how to build complex applications](./../develop/introduction.md) on top of Swarm's underlying infrastructure. --- ## Build from Source Bee is written using the [Go](https://go.dev) language. You may build the Bee client software directly from the [source](https://github.com/ethersphere/bee). Prerequisites for installing directly from source are: - **go** - download the latest release from [go.dev](https://go.dev/dl). - **git** - download from [git-scm.com](https://git-scm.com/). - **make** - [make](https://www.gnu.org/software/make/) is usually included by default in most UNIX operating systems, and can be installed and used on almost any other operating system where it is not included by default. ## Build steps 1. Clone the repository: ```bash git clone https://github.com/ethersphere/bee cd bee ``` 2. Use `git` to find the latest release: ```bash git describe --tags ``` 3. Checkout the required version: ```bash git checkout v2.8.1 ``` 4. Build the binary: ```bash make binary ``` 5. Check that you are able to run the `bee` command. Success can be verified by running: ```bash dist/bee version ``` ``` 2.8.1 ``` 6. (optional) Additionally, you may also like to move the Bee binary to somewhere in your `$PATH` ```bash sudo cp dist/bee /usr/local/bin/bee ``` --- ## Connectivity To fully connect to the swarm, your Bee node needs to be able to both send and receive messages from the outside world. Normally, your router will not allow other IPs on the Internet to connect, unless you have initiated the connection. Bees welcome newcomers in the swarm, as long as they play by the rules! If a node misbehaves, we will simply add it to a list of blocked nodes and refuse future connections from them. Here at Swarm, every Bee counts! To make sure all Bees can join the swarm, below you will find a detailed guide to navigating your way through your network and making it out into the wild so you can buzz around fellow bees and maximize your chances of earning xBZZ. If you still have problems, please join us in our [Discord server](https://discord.gg/kHRyMNpw7t) and we'll help you find the way! 🐝 🐝 🐝 🐝 🐝 :::warning To ensure your Bee has the best chance of participating in the swarm, you must ensure your Bee is able to handle **both incoming and outgoing connections from the global Internet to its p2p port (`1634` by default)**. See below for a detailed guide on how to make sure this is the case, or for the 1337: check your `http://localhost:1633/addresses` to see which public IP and port libp2p is advertising and verify its connectivity to the rest of the Internet! You may need to alter your Bee node's `nat-addr` configuration. 🤓 ::: ## Networking Basics In a network, each computer is assigned an IP address. Each IP address is then subdivided into thousands of _sockets_ or _ports_, each of which has an incoming and outgoing component. In a completely trusted network of computers, any connections to or from any of these ports are allowed. However, to protect ourselves from nefarious actors when we join the wider Internet, it is sometimes important to filter this traffic so that some of these ports are off limits to the public. In order to allow messages to our p2p port from other Bee nodes that we have previously not connected, we must ensure that our network is set up to receive incoming connections (on port `1634` by default). :::danger There are also some ports which you should never expose to the outside Internet. Make sure that your `api-addr` (default `1633`) is never exposed to the internet. It is good practice to employ one or more firewalls that block traffic on every port except for those you are expecting to be open. If you do not use a firewall, make sure to change the default `api-addr` from `1633` to `127.0.0.1:1633` so that it is not publicly exposed. ::: ### Your IP Address When you connect to the Internet, you are assigned a unique number called an IP Address. IP stands for **Internet Protocol**. The most prevalent IP version used is _still_ the archaic [IPv4](https://en.wikipedia.org/wiki/IPv4) which was invented way back in 1981. IPv6 is available but not well used. Due to the mitigation of the deficiencies inherent in the IPv4 standard, some complications may arise. ### Datacenters and Computers Connected Directly to the Internet If you are renting space in a datacenter, the chances are that your computer will be connected directly to the real Internet. This means that the IP of your networking interface will be directly set to be the same as your public IP. You can investigate this by running: ```bash ifconfig ``` or ```bash ip address ``` Your output should contain something like: ``` eth0: flags=4163 mtu 1500 inet 178.128.196.191 netmask 255.255.240.0 broadcast 178.128.207.255 ``` Here we can see our computer's **public IP address** `178.128.196.191`. This is the address that is used by other computers we connect to over the Internet. We can verify this using a third party service such as _icanhazip_ or _ifconfig_. ```bash curl icanhazip.com --ipv4 ``` or ```bash curl ifconfig.co --ipv4 ``` The response may contain something like: ``` 178.128.196.191 ``` With Bee running, try to connect to your Bee's p2p port using the public IP address from another computer: ```bash nc -zv 178.128.196.191 1634 ``` If you have success, congratulations! If this still doesn't work for you, see the last part of _Manual: Configure Your Router and Bee_ section below, as you may need to configure your `nat-addr`. ### Home, Commercial and Business Networks and Other Networks Behind NAT To address the [scarcity of IP numbers](https://en.wikipedia.org/wiki/IPv4_address_exhaustion), Network Address Translation (NAT) was implemented. This approach creates a smaller, private network which many devices connect to in order to share a public IP address. Traffic destined for the Internet at large is then mediated by another specialised computer. In the cases of the a home network, this computer is the familiar home router, normally also used to provide a WiFi network. If we run the above commands to find the computer's IP in this scenario, we will see a different output. ```bash ip address ``` ``` en0: flags=8863 mtu 1500 ... inet 192.168.0.10 netmask 0xffffff00 broadcast 192.168.0.255 ... ``` Here we can see that, instead of the public IP address, we can see that our computer's IP address is `192.168.0.10`. This is part of the IP address space that the Internet Engineering Task Force has designated for [private networks](https://en.wikipedia.org/wiki/Private_network). As this IP won't work on the global Internet, our router remembers that our computer has been assigned this IP. It then uses _Network Address Translation_ (NAT) to modify all requests from our computer to another computer somewhere in the Internet. As the requests pass through the router it changes our local IP to the public IP of the router, and vice versa when the responses are sent back, from the public IP to the local one. ## Navigating Through the NAT The presence of NAT presents two problems for p2p networking. The first is that it can be difficult for programs running on our computer to know our real public IP as it is not explicitly known by our computer's networking interface, which is configured with a private network IP. This is a relatively easy problem to solve as we can simply discover our public IP and then specify it in Bee's configuration, or indeed determine it using other means. The second issue is that our router has only 65535 ports to expose to the public network. However, *each device on your private network is capable of exposing 65535 ports*. To the global Internet, it appears that there is only one set of ports to connect to, whereas, in actual fact, there is a full set of ports for each of the devices which are connected to the private network. To solve this second problem, routers commonly employ an approach known as _port forwarding_. Bee's solution to these problems come in two flavours, automatic and manual. ### Automatic: Universal Plug and Play (UPnP) UPnP is a protocol designed to simplify the administration of NAT and port forwarding for the end user by providing an API from which software running within the network can use to ask the router for the public IP and to request for ports to be forwarded to the private IP of the computer running the software. :::danger UPnP is a security risk! UPnP is a security risk as it allows any host or process inside (sometimes also outside) your network to open arbitrary ports which may be used to transfer malicious traffic, for example a [RAT](https://en.wikipedia.org/wiki/Remote_desktop_software#RAT). UPnP can also be used to determine your IP, and in the case of using ToR or a VPN, your _real_ public IP. We urge you to disable UPnP on your router and use manual port forwarding as described below. ::: Bee will use UPnP to determine your public IP, which is required for various internal processes. In addition to this, a request will be sent to your router to ask it to forward a random one of its ports, which are exposed directly to the Internet, to the Bee p2p port (default `1634`) which your computer is exposing only to the private network. Doing this creates a tunnel through which other Bees may connect to your computer safely. If you start your Bee node in a private network with UPnP available, the output of the addresses endpoint of your API will look something like this: ```json [ "/ip4/127.0.0.1/tcp/1634/p2p/16Uiu2HAm5zcoBFWmqjDTwGy9RXepBFF8idy6Pr312obMwwxdJSUP", "/ip4/192.168.0.10/tcp/1634/p2p/16Uiu2HAm5zcoBFWmqjDTwGy9RXepBFF8idy6Pr312obMwwxdJSUP", "/ip6/::1/tcp/1634/p2p/16Uiu2HAm5zcoBFWmqjDTwGy9RXepBFF8idy6Pr312obMwwxdJSUP", "/ip4/86.98.94.9/tcp/20529/p2p/16Uiu2HAm5zcoBFWmqjDTwGy9RXepBFF8idy6Pr312obMwwxdJSUP" ] ``` Note that the port in the external [multiaddress](https://docs.libp2p.io/concepts/addressing/) is the router's randomly selected `20529` which is forwarded by the router to `192.168.0.10:1634`. These addresses in this multiaddress are also known as the underlay addresses. ### Manual: Configure Your Router and Bee Inspecting the underlay addresses in the output of the addresses endpoint of our API, we can see addresses only for _localhost_ `127.0.0.1` and our _private network IP_ `192.168.0.10`. Bee must be having trouble navigating our NAT. ```json [ "/ip4/127.0.0.1/tcp/1634/p2p/16Uiu2HAm8Hs91MzWuXfUyKrYaj3h8K8gzvRqzSK5gP9TNCwypkJB", "/ip4/192.168.0.10/tcp/1634/p2p/16Uiu2HAm8Hs91MzWuXfUyKrYaj3h8K8gzvRqzSK5gP9TNCwypkJB", "/ip6/::1/tcp/1634/p2p/16Uiu2HAm8Hs91MzWuXfUyKrYaj3h8K8gzvRqzSK5gP9TNCwypkJB" ] ``` To help fix the first problem, let's determine our public IP address. ```bash curl icanhazip.com ``` ``` 86.98.94.9 ``` Now we can simply supply this IP in our Bee configuration on startup. Solving our second problem is a little more difficult as we will need to interact with our router's firmware, which is a little cranky. Each router is different, but the concept is usually the same. Log in to your router by navigating your browser to your router's configuration user interface, usually at [http://192.168.0.1](http://192.168.0.1). You will need to log in with a password. Sadly, passwords are often left to be the defaults, which can be found readily on the Internet. Once logged in, find the interface to set up port forwarding. The [Port Forward](https://portforward.com/router.htm) website provides some good information, or you may refer to your router manual or provider. Here, we will then set up a rule that forwards port `1634` of our private IP address `192.168.0.10` to the same port `1634` of our public IP. Now, when requests arrive at our public address `86.98.94.9:1634` they are modified by our router and forwarded to our private IP and port `192.168.0.10:1634`. Sometimes this can be a little tricky, so let's verify we are able to make a TCP connection using [netcat](https://nmap.org/ncat/). First, with Bee **not** running, let's set up a simple TCP listener using Netcat on the same machine we would like to run Bee on. ```bash nc -l 0.0.0.0 1634 ``` ```bash nc -zv 86.98.94.9 1634 ``` ``` Connection to 86.98.94.9 port 1834 [tcp/*] succeeded! ``` Success! ✨ If this didn't work for you, check out our Debugging Connectivity guide below. If it did, let's start our Bee node with the `--nat-addr` configured. ```bash bee start --nat-addr 86.98.94.9:1634 ``` Checking our addresses endpoint again, we can now see that Bee has been able to successfully assign a public address! Congratulations, your Bee is now connected to the outside world! ```json [ "/ip4/127.0.0.1/tcp/1634/p2p/16Uiu2HAm8Hs91MzWuXfUyKrYaj3h8K8gzvRqzSK5gP9TNCwypkJB", "/ip4/192.168.0.10/tcp/1634/p2p/16Uiu2HAm8Hs91MzWuXfUyKrYaj3h8K8gzvRqzSK5gP9TNCwypkJB", "/ip6/::1/tcp/1634/p2p/16Uiu2HAm8Hs91MzWuXfUyKrYaj3h8K8gzvRqzSK5gP9TNCwypkJB", "/ip4/86.98.94.9/tcp/1634/p2p/16Uiu2HAm8Hs91MzWuXfUyKrYaj3h8K8gzvRqzSK5gP9TNCwypkJB" ] ``` :::info If you are regularly connecting and disconnecting to a network, you may also want to use your router's firmware to configure the router to reserve and only assign the same local network IP from its DHCP pool to your computer's MAC address. This will ensure that your Bee seamlessly connects when you rejoin the network! ::: ### Using multiple P2P transports (TCP, WS, WSS) A Bee node can expose more than one transport for peer-to-peer communication. By default, nodes use the TCP-based libp2p transport, but Secure WebSocket (`WSS`) transport can also be enabled. To enable WSS support, set: ```yaml p2p-wss-enable: true ```` When enabled, Bee listens for Secure WebSocket connections on `p2p-wss-addr` (default `:1635`). In most cases the remaining WSS and AutoTLS options can be left at their default values: ```yaml p2p-wss-addr: ":1635" nat-wss-addr: "" autotls-domain: libp2p.direct autotls-registration-endpoint: https://registration.libp2p.direct autotls-ca-endpoint: https://acme-v02.api.letsencrypt.org/directory ``` A configuration using both TCP and WSS transports may look like: ```yaml p2p-addr: :1634 p2p-wss-enable: true p2p-wss-addr: :1635 nat-addr: 1.2.3.4:1634 nat-wss-addr: node.example.com:443 ``` In this example: * `p2p-addr` defines the local TCP listening address. * `p2p-wss-addr` defines the local Secure WebSocket listening address. * `nat-addr` is the public address advertised to peers for TCP connections. * `nat-wss-addr` is the public address advertised to peers for WSS connections. If WSS is enabled, the WSS port must be reachable by peers. This means the port should be open in your firewall, exposed by your container or host configuration, and permitted by your network if outbound connections are restricted. When specifying `nat-addr` or `nat-wss-addr`, the value must be a valid `host:port` pair. For example: ```yaml nat-addr: 1.2.3.4:1634 nat-wss-addr: node.example.com:443 ``` Values missing either the host or port or otherwise misformed addresses are considered invalid and will prevent the node from starting. ### Troubleshooting Connectivity The above guide navigates your NAT, but there are still a few hurdles to overcome. To make sure there is a clear path from your computer to the outside world, let's follow our Bee's journey from the inside out. Let's set up a netcat listener on all interfaces on the computer we'd like to run Bee on as we have above. ```bash nc -l 0.0.0.0 1634 ``` Now, let's verify we're able to connect to netcat by checking the connection from our local machine. ```bash nc -zv 127.0.0.1 1634 ``` ``` Connection to 127.0.0.1 port 1634 [tcp/*] succeeded! ``` This should be a no brainer, the connection between localhost in not normally mediated. If there is a problem here, the problem is with some other software running on your operating system or your operating system itself. Try a different port, such as `1734` and turning off any unnecessary software. If this doesn't work, you may need to try a different operating system environment. Please get in touch and we'll try to help! If we were successful, let's move on to the next stage. :::info If you are not able to get access to some firewall settings, or otherwise debug incoming connectivity, don't worry! All is not lost. Bee can function just fine with just outgoing connections. However, if you can, it is worth the effort to allow incoming connections, as the whole swarm will benefit from the increased connectivity. ::: Let's find out what our IP looks like to the Internet. ```bash curl icanhazip.com ``` ``` 86.98.94.9 ``` Now try to connect to your port using the global IP. ```bash nc -zv 86.98.94.9 1634 ``` If this is successful, our Bee node's path is clear! If not, we can try a few things to make sure there are no barriers stopping us from getting through. 1. Check your computer's firewall. Sometimes your computer is configured to prevent connections. If you are on a private network mediated by NAT, you can check if this is the problem by trying to connect from another device on your network using the local IP `nc -zv 192.168.0.10 1634`. Ubuntu uses [UFW](https://help.ubuntu.com/community/UFW), MacOS can be configured using the _Firewall_ tab in the _Security & Privacy_ section of _System Preferences_. Windows uses [Defender Firewall](https://support.microsoft.com/en-US/windows/security/windows-security/firewall-and-network-protection-in-the-windows-security-app). For each of these firewalls, set a special rule to allow UDP and TCP traffic to pass through on port `1634`. You may want to limit this traffic to the Bee application only. 2. Check your ingress' firewall. For a datacenter hired server, this configuration will often take place in somewhere in the web user interface. Refer to your server hosting provider's documentation to work out how to open ports to the open Internet. Ensure that both TCP and UDP traffic are allowed. Similarly, if you are connecting from within a private network, you may find that the port is blocked by the router. Each router is different, so consult your router's firmware documentation to make sure there are no firewalls in place blocking traffic on your Bee's designated p2p port. You may check this using netcat by trying to connect using your computer's public IP, as above `nc -zv 86.98.94.9 1634`. 3. Docker Docker adds another level of complexity. To debug docker connectivity issues, we may use netcat as above to check port connections are working as expected. Double check that you are exposing the right ports to your local network, either by using the command line flags or in your docker-compose.yaml. You should be able to successfully check the connection locally using eg. `nc -zv localhost 1634` then follow instructions above to make sure your local network has the correct ports exposed to the Internet. 3. Something else entirely? Networking is a complex topic, but it keeps us all together. If you still can't connect to your Bee, get in touch via the [official node operator's Discord channel](https://discord.gg/kHRyMNpw7t) and we'll do our best to get you connected. In Swarm, no Bee is left behind. --- ## Docker Install The following is a guide for installing a Bee node using Docker. Docker images for Bee are hosted at [Docker Hub](https://hub.docker.com/r/ethersphere/bee). Using Docker to operate your Bee node offers many benefits, such as ease of deployment and consistency across environments. :::caution In the examples below we specify the exact image version as 2.8.1. It's recommended to only use the exact version number tags. Make sure to check that you're on the latest version of Bee by reviewing the tags for Bee on [Docker Hub](https://hub.docker.com/r/ethersphere/bee/tags), and replace 2.8.1 in the commands below if there is a newer full release. ::: :::warning Note that in all the examples below we map the Bee API to 127.0.0.1 (localhost), since we do not want to expose our Bee API endpoint to the public internet, as that would allow anyone to control our node. Make sure you do the same, and it's also recommended to use a firewall to protect access to your node(s). ::: :::info This guide sets options using environment variables as part of the Docker startup commands such as `-e BEE_API_ADDR=":1633"`, however there are [several other methods available for configuring options](./../working-with-bee/configuration.md). ::: ## Node setup process This section will guide you through setting up and running a single full Bee node using Docker. In the guide, we use a single line command for running our Bee node, with the Bee config options being set through environment variables, and a single volume hosted for our node's data. ### Start node ```bash docker run -d --name bee-1 \ --restart always \ -p 127.0.0.1:1633:1633 \ -p 1634:1634 \ -e BEE_API_ADDR=":1633" \ -e BEE_FULL_NODE="true" \ -e BEE_SWAP_ENABLE="true" \ -e BEE_PASSWORD="flummoxedgranitecarrot" \ -e BEE_BLOCKCHAIN_RPC_ENDPOINT="https://xdai.fairdatasociety.org" \ -v bee-1:/home/bee/.bee \ ethersphere/bee:2.8.1 start ``` Here is the same command in a single line in case you run into issues with the line breaks in the command above: ```bash docker run -d --name bee-1 --restart always -p 127.0.0.1:1633:1633 -p 1634:1634 -e BEE_API_ADDR=":1633" -e BEE_FULL_NODE="true" -e BEE_SWAP_ENABLE="true" -e BEE_PASSWORD="flummoxedgranitecarrot" -e BEE_BLOCKCHAIN_RPC_ENDPOINT="https://xdai.fairdatasociety.org" -v bee-1:/home/bee/.bee ethersphere/bee:2.8.1 start ``` #### Command explained: - **`-d`**: Runs the container in the background. - **`--restart always`**: Sets the [restart policy](https://docs.docker.com/engine/containers/start-containers-automatically/) for the container to `always`. - **`--name bee-1`**: Names the container `bee-1`. - **`-p 127.0.0.1:1633:1633`**: Exposes the API on port 1633, only accessible locally. - **`-p 1634:1634`**: Exposes the P2P port 1634 to the public. - **`-e BEE_API_ADDR=":1633"`**: Sets the Bee API to use port 1633. - **`-e BEE_FULL_NODE="true"`**: Runs as a full node. - **`-e BEE_SWAP_ENABLE="true"`**: Enables the SWAP protocol for payments. - **`-e BEE_PASSWORD="flummoxedgranitecarrot"`**: Sets the keystore password, make sure to replace with your own. - **`-e BEE_BLOCKCHAIN_RPC_ENDPOINT="https://xdai.fairdatasociety.org"`**: Connects to the Gnosis Chain. - **`-v bee-1:/home/bee/.bee`**: Persists node data in the `bee-1` volume. - **`ethersphere/bee:2.8.1 start`**: Runs Bee version 2.8.1 and starts the node. This setup runs the Bee node in a container, with full node functionality, SWAP enabled, and connections to the Gnosis blockchain for chequebook and postage stamp management, while persisting its data using a volume. :::info We have included the password directly in the start command as an environment variable with `-e BEE_PASSWORD="flummoxedgranitecarrot"`. You may wish to use a password file instead, which can be set with the `BEE_PASSWORD_FILE` command. However this will likely require some modifications on your host machine, the details of which will vary from system to system. ::: ```bash docker ps ``` If everything is set up correctly, you should see your Bee node listed: ```bash CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 37f4ad8b4060 ethersphere/bee:2.8.1 "bee start" 6 seconds ago Up 5 seconds 127.0.0.1:1633->1633/tcp, 0.0.0.0:1634->1634/tcp, :::1634->1634/tcp bee-1 ``` And check the logs: ```bash docker logs -f bee-1 ``` The output should contain a line which prints a message notifying you of the minimum required xDAI for running a node as well as the address of your node. Copy the address and save it for use in the next section. ```bash "time"="2024-09-24 22:06:51.363708" "level"="warning" "logger"="node/chequebook" "msg"="cannot continue until there is at least min xDAI (for Gas) available on address" "min_amount"="0.0003576874793" "address"="0x91A7e3AC06020750D32CeffbEeFD55B4c5e42bd6" ``` You can use `Ctrl + C` to exit the logs. Before moving on to funding, stop your node: ```bash docker stop bee-1 ``` And let's confirm that it has stopped: ```bash docker ps ``` We can confirm no Docker container processes are currently running. ```bash CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ```` ### Fund node Check the logs from the previous step. Look for the line which says: ``` "time"="2024-09-24 18:15:34.520716" "level"="info" "logger"="node" "msg"="using ethereum address" "address"="0x1A801dd3ec955E905ca424a85C3423599bfb0E66" ``` That address is your node's address on Gnosis Chain which needs to be funded with xDAI and xBZZ. Copy it and save it for the next step. xDAI is widely available from many different centralized and decentralized exchanges, just make sure that you are getting xDAI on Gnosis Chain, and not DAI on some other chain. See [this page](https://www.ethswarm.org/get-bzz) for a list of resources for getting xBZZ (again, make certain that you are getting the Gnosis Chain version, and not BZZ on Ethereum). After acquiring some xDAI and some xBZZ, send them to the address you copied above. ***How Much to Send?*** Only a very small amount of xDAI is needed to get started, 0.1 is more than enough. You can start with just 2 or 3 xBZZ for uploading small amounts of data, but you will need at least 10 xBZZ if you plan on staking. ### Initialize full node After you have a small amount of xDAI in your node's Gnosis Chain address, you can now restart your node using the same command as before so that it can issue the required smart contract transactions and also sync data. ```bash docker start bee-1 ``` Let's check the logs to see what's happening: ```bash docker logs -f bee-1 ``` Your logs should look something like this: ```bash Welcome to Swarm.... Bzzz Bzzzz Bzzzz \ / \ o ^ o / \ ( ) / ____________(%%%%%%%)____________ ( / / )%%%%%%%( \ \ ) (___/___/__/ \__\___\___) ( / /(%%%%%%%)\ \ ) (__/___/ (%%%%%%%) \___\__) /( )\ / (%%%%%) \ (%%%) ! DISCLAIMER: This software is provided to you "as is", use at your own risk and without warranties of any kind. It is your responsibility to read and understand how Swarm works and the implications of running this software. The usage of Bee involves various risks, including, but not limited to: damage to hardware or loss of funds associated with the Ethereum account connected to your node. No developers or entity involved will be liable for any claims and damages associated with your use, inability to use, or your interaction with other nodes or the software. "time"="2026-07-07 16:52:59.641444" "level"="info" "logger"="node" "msg"="bee version" "version"="2.8.1-7cf53193" "time"="2026-07-07 16:52:59.793257" "level"="info" "logger"="node" "msg"="swarm public key" "public_key"="02d8d7e1ca6b3b43653ae27e35a375dd74e3ce2f40587fd264bc7268ed918650ab" "time"="2026-07-07 16:53:00.087534" "level"="info" "logger"="node" "msg"="pss public key" "public_key"="02aaae4ede42f47f48aa5182df4b94039ca71254f44ebc5383d5a67f71fe7e6156" "time"="2024-09-24 22:21:04.686464" "level"="info" "logger"="node" "msg"="using ethereum address" "address"="0x8288F1c8e3dE7c3bf42Ae67fa840EC61481D085e" "time"="2024-09-24 22:21:04.700711" "level"="info" "logger"="node" "msg"="using overlay address" "address"="22dc155fe072e131449ec7ea2f77de16f4735f06257ebaa5daf2fdcf14267fd9" "time"="2024-09-24 22:21:04.700741" "level"="info" "logger"="node" "msg"="starting with an enabled chain backend" "time"="2024-09-24 22:21:05.298019" "level"="info" "logger"="node" "msg"="connected to blockchain backend" "version"="Nethermind/v1.28.0+9c4816c2/linux-x64/dotnet8.0.8" "time"="2024-09-24 22:21:05.485287" "level"="info" "logger"="node" "msg"="using chain with network network" "chain_id"=100 "network_id"=1 "time"="2024-09-24 22:21:05.498845" "level"="info" "logger"="node" "msg"="starting debug & api server" "address"="[::]:1633" "time"="2024-09-24 22:21:05.871498" "level"="info" "logger"="node" "msg"="using default factory address" "chain_id"=100 "factory_address"="0xC2d5A532cf69AA9A1378737D8ccDEF884B6E7420" "time"="2024-09-24 22:21:06.059179" "level"="info" "logger"="node/chequebook" "msg"="no chequebook found, deploying new one." "time"="2024-09-24 22:21:07.386747" "level"="info" "logger"="node/chequebook" "msg"="deploying new chequebook" "tx"="0x375ca5a5e0510f8ab307e783cf316dc6bf698c15902a080ade3c1ea0c6059510" "time"="2024-09-24 22:21:19.101428" "level"="info" "logger"="node/transaction" "msg"="pending transaction confirmed" "sender_address"="0x8288F1c8e3dE7c3bf42Ae67fa840EC61481D085e" "tx"="0x375ca5a5e0510f8ab307e783cf316dc6bf698c15902a080ade3c1ea0c6059510" "time"="2024-09-24 22:21:19.101450" "level"="info" "logger"="node/chequebook" "msg"="chequebook deployed" "chequebook_address"="0x66127e4393956F11947e9f54599787f9E455173d" "time"="2024-09-24 22:21:19.506515" "level"="info" "logger"="node" "msg"="using datadir" "path"="/home/bee/.bee" "time"="2024-09-24 22:21:19.518258" "level"="info" "logger"="migration-RefCountSizeInc" "msg"="starting migration of replacing chunkstore items to increase refCnt capacity" "time"="2024-09-24 22:21:19.518283" "level"="info" "logger"="migration-RefCountSizeInc" "msg"="migration complete" "time"="2024-09-24 22:21:19.566160" "level"="info" "logger"="node" "msg"="starting reserve repair tool, do not interrupt or kill the process..." "time"="2024-09-24 22:21:19.566232" "level"="info" "logger"="node" "msg"="removed all bin index entries" "time"="2024-09-24 22:21:19.566239" "level"="info" "logger"="node" "msg"="removed all chunk bin items" "total_entries"=0 "time"="2024-09-24 22:21:19.566243" "level"="info" "logger"="node" "msg"="counted all batch radius entries" "total_entries"=0 "time"="2024-09-24 22:21:19.566247" "level"="info" "logger"="node" "msg"="parallel workers" "count"=20 "time"="2024-09-24 22:21:19.566271" "level"="info" "logger"="node" "msg"="migrated all chunk entries" "new_size"=0 "missing_chunks"=0 "invalid_sharky_chunks"=0 "time"="2024-09-24 22:21:19.566294" "level"="info" "logger"="migration-step-04" "msg"="starting sharky recovery" "time"="2024-09-24 22:21:19.664643" "level"="info" "logger"="migration-step-04" "msg"="finished sharky recovery" "time"="2024-09-24 22:21:19.664728" "level"="info" "logger"="migration-step-05" "msg"="start removing upload items" "time"="2024-09-24 22:21:19.664771" "level"="info" "logger"="migration-step-05" "msg"="finished removing upload items" "time"="2024-09-24 22:21:19.664786" "level"="info" "logger"="migration-step-06" "msg"="start adding stampHash to BatchRadiusItems, ChunkBinItems and StampIndexItems" "time"="2024-09-24 22:21:19.664837" "level"="info" "logger"="migration-step-06" "msg"="finished migrating items" "seen"=0 "migrated"=0 "time"="2024-09-24 22:21:19.664897" "level"="info" "logger"="node" "msg"="waiting to sync postage contract data, this may take a while... more info available in Debug loglevel" ``` Your node will take some time to finish [syncing postage contract data](https://docs.ethswarm.org/docs/develop/tools-and-features/buy-a-stamp-batch/) as indicated by the final line: ```bash "msg"="waiting to sync postage contract data, this may take a while... more info available in Debug loglevel" ``` You may need to wait 5 - 10 minutes for your node to finish syncing in this step. Eventually you will be able to see when your node finishes syncing, and the logs will indicate your node is starting in full node mode: ```bash "time"="2024-09-24 22:30:19.154067" "level"="info" "logger"="node" "msg"="starting in full mode" "time"="2024-09-24 22:30:19.155320" "level"="info" "logger"="node/multiresolver" "msg"="name resolver: no name resolution service provided" "time"="2024-09-24 22:30:19.341032" "level"="info" "logger"="node/storageincentives" "msg"="entered new phase" "phase"="reveal" "round"=237974 "block"=36172090 "time"="2024-09-24 22:30:33.610825" "level"="info" "logger"="node/kademlia" "msg"="disconnected peer" "peer_address"="6ceb30c7afc11716f866d19b7eeda9836757031ed056b61961e949f6e705b49e" ``` Your node will now begin syncing chunks from the network, this process can take several hours. You check your node's progress with the `/status` endpoint: ```bash curl -s http://localhost:1633/status | jq ``` ```bash { "overlay": "22dc155fe072e131449ec7ea2f77de16f4735f06257ebaa5daf2fdcf14267fd9", "proximity": 256, "beeMode": "full", "reserveSize": 686217, "reserveSizeWithinRadius": 321888, "pullsyncRate": 497.8747754074074, "storageRadius": 11, "connectedPeers": 148, "neighborhoodSize": 4, "batchCommitment": 74510761984, "isReachable": false, "lastSyncedBlock": 36172390 } ``` We can see that our node has not yet finished syncing chunks since the `pullsyncRate` is around 497 chunks per second. Once the node is fully synced, this value will go to zero. It can take several hours for syncing to complete, but we do not need to wait until our node is fully synced before staking, so we can move directly to the next step. ### Stake node You can use the following command to stake 10 xBZZ: ```bash curl -XPOST localhost:1633/stake/100000000000000000 ``` If the staking transaction is successful a `txHash` will be returned: ``` {"txHash":"0x258d64720fe7abade794f14ef3261534ff823ef3e2e0011c431c31aea75c2dd5"} ``` We can also confirm that our node has been staked with the `/stake` endpoint: ```bash curl localhost:1633/stake ``` The results will be displayed in PLUR units (1 PLUR is equal to 1e-16 xBZZ). If you have properly staked the minimum 10 xBZZ, you should see the output below: ```bash {"stakedAmount":"100000000000000000"} ``` Congratulations! You have now installed your Bee node and successfully connected it to the network as a full staking node. Your node will now be in the process of syncing chunks from the network. Once it is fully synced, your node will finally be eligible for earning staking rewards. ### Set Target Neighborhood When installing your Bee node it will automatically be assigned a neighborhood. However, when running a full node with staking there are benefits to periodically updating your node's neighborhood. Learn more about why and how to set your node's target neighborhood [here](./set-target-neighborhood.md). ### Logs and monitoring Docker provides convenient built-in tools for logging and monitoring your node, which you've already encountered if you've read through earlier sections of this guide. For a more detailed guide, [refer to the section on logging](./../working-with-bee/logs-and-files.md). **Viewing node logs:** To monitor your node’s logs in real-time, use the following command: ```bash docker logs -f bee-1 ``` This command will continuously output the logs of your Bee node, helping you track its operations. The `-f` flag ensures that you see new log entries as they are written. Press `Ctrl + C` to stop following the logs. You can read more about how Docker manages container logs [in their official docs](https://docs.docker.com/reference/cli/docker/container/logs/). **Checking the Node's status with the Bee API** To check your node's status as a staking node, we can use the `/redistributionstate` endpoint: ```bash curl -s http://localhost:1633/redistributionstate | jq ``` Below is the output for a node which has been running for several days: ```bash { "minimumGasFunds": "11080889201250000", "hasSufficientFunds": true, "isFrozen": false, "isFullySynced": true, "phase": "claim", "round": 212859, "lastWonRound": 207391, "lastPlayedRound": 210941, "lastFrozenRound": 210942, "lastSelectedRound": 212553, "lastSampleDuration": 491687776653, "block": 32354719, "reward": "1804537795127017472", "fees": "592679945236926714", "isHealthy": true } ``` For a complete breakdown of this output, check out [this section in the Bee docs](https://docs.ethswarm.org/docs/bee/working-with-bee/bee-api#redistributionstate). You can read more other important endpoints for monitoring your Bee node in the [official Bee docs](https://docs.ethswarm.org/docs/bee/working-with-bee/bee-api), and you can find complete information about all available endpoints in [the API reference docs](https://docs.ethswarm.org/api/). **Stopping Your Node** To gracefully stop your Bee node, use the following command: ```bash docker stop bee-1 ``` Replace `bee-1` with the name of your node if you've given it a different name. ## Back Up Keys Once your node is up and running, make sure to [back up your keys](./../working-with-bee/backups.md). ## Getting help The CLI has documentation built-in. Running `bee` gives you an entry point to the documentation. Running `bee start -h` from within your Docker container or `bee start --help` will tell you how you can configure your Bee node via the command line arguments. You may also check out the [configuration guide](./../working-with-bee/configuration.md), or simply run your Bee terminal command with the `--help` flag, eg. `bee start --help` or `bee --help`. ## Next Steps to Consider ### Access the Swarm If you'd like to start uploading or downloading files to Swarm, [start here](./../../develop/introduction.md). ### Explore the API The [Bee API](./../working-with-bee/bee-api.md) is the primary method for interacting with Bee and getting information about Bee. After installing Bee and getting it up and running, it's a good idea to start getting familiar with the API. ### Run a hive! If you would like to run a hive of many Bees, check out the [hive operators](./hive.md) section for information on how to operate and monitor many Bees at once. ### Start building DApps on Swarm If you would like to start building decentralised applications on Swarm, check out our section for [developing with Bee](./../../develop/introduction.md). --- ## Fund Your Node ## Overview Bee nodes require **xDAI** (for gas fees) and **xBZZ** (for storage and bandwidth) to function properly. The amount needed depends on your node type and use case. ### xDAI is Required For: - **Buying Postage Stamps** ([Uploading Data](./../../develop/tools-and-features/buy-a-stamp-batch.md)) - **Stake Management Transactions** ([Staking](./../working-with-bee/staking.md)) - **Storage Incentives Transactions** ([Redistribution Game](./../../concepts/incentives/redistribution-game.md)) - **Chequebook Deployment** ([Bandwidth Payments](./../../concepts/incentives/bandwidth-incentives.md)) ### xBZZ is Required For: - **Buying Postage Stamps** (scales with data size and duration) - **Staking** (Minimum **10 xBZZ**, **20 xBZZ** for reserve doubling) - **Bandwidth Payments** (~**0.5 xBZZ per GB downloaded**) ## Token Amounts by Use Case | **Use Case** | **Node Type** | **xDAI Required** | **xBZZ Required** | |-------------|--------------|------------------|------------------| | Free tier downloads | Ultra-Light, Light, Full | None | None | | Downloading beyond free tier | Light, Full | None |Scales with volume—start with ~0.1 xBZZ, increase as needed | | Uploading | Light, Full | None | Scales with volume—start with ~0.1 xBZZ, increase as needed | | Purchasing Postage Stamp Batches| Light, Full | < 0.01 xDAI / tx | Scales with volume & duration. Can start with ~0.2 xBZZ for small uploads. | | Staking | Full | < 0.01 xDAI / tx | 10 xBZZ (minimum) | | Storage Incentives Transactions | Full | < 0.01 xDAI / tx - needs topups over time since these are reoccurring transactions | None | | Bandwidth Payments | Light, Full | None | Scales with bandwidth (~0.5 xBZZ/GB downloaded) | | Chequebook Deployment | Light, Full | < 0.001 xDAI | None | ## Getting Tokens ### How to Get xDAI - **Free xDAI Faucets**: You may try one of the [Gnosis Chain faucets](https://docs.gnosischain.com/tools/Faucets) listed in the official Gnosis Chain documentation, however the amount offered may not meet your needs. - **Purchasing xDAI**: You can also purchase xDAI from [various exchanges](https://docs.gnosischain.com/about/tokens/xdai) listed in the Gnosis Chain documentation. xDAI is also widely available on most major cryptocurrency exchanges. :::warning Make sure that you are withdrawing the Gnosis Chain version of xDAI, as xDAI has been bridged to several other chains as well. ::: - **Bridging From Ethereum**: If you already have xDAI on Ethereum, you can also consider using the [Gnosis Chain bridge](https://bridge.gnosischain.com/) to transfer it over to Gnosis Chain. ### How to Get xBZZ - **Buying xBZZ**: xBZZ can be purchased from a variety of [centralized and decentralized exchanges](https://www.ethswarm.org/get-bzz#how-to-get-bzz) listed on the official Ethswarm.org website. ### Getting Testnet Tokens (Sepolia ETH & sBZZ) - **Sepolia ETH**: Try [these faucets](https://faucetlink.to/sepolia). - **sBZZ**: Buy on [Uniswap](https://app.uniswap.org/swap?outputCurrency=0x543dDb01Ba47acB11de34891cD86B675F04840db&inputCurrency=ETH) (ensure **Sepolia testnet** is selected in MetaMask and **Testnet mode** is enabled in the Uniswap web app settings). ## Node Wallet & Chequebook - **Wallet Creation**: A Gnosis Chain wallet is auto-created when you install Bee. - **Chequebook Deployment**: A chequebook contract will be automatically deployed when a Bee node is configured to run as a light or full node and has been funded with sufficient xDAI to pay for the chequebook deployment transaction. Required for bandwidth payments. - **Wallet Access**: Located in `keys/` in Bee's `data-dir` (importable to MetaMask). Also requires a password which is specified through your node's configuration (either passed directly with the `password` option or as a password file specified with the `password-file` option). ## Funding Your Wallet In order to fund your wallet, first you need to identify your wallet address. The easiest way to do so is to first start your Bee node in ultra-light mode (Bee will start in ultra-light mode when started with the default settings) and then query the Bee API to find your address: ```bash curl -s localhost:1633/addresses | jq .ethereum ``` ```bash "0x9a73f283cd9212b99b5e263f9a81a0ddc847cd93" ``` Fund your node with the appropriate amount of xDAI and xBZZ based on the recommended amounts specified in [the chart above](./fund-your-node.md#token-amounts-by-use-case). *For support, ask in the [Develop on Swarm](https://discord.com/channels/799027393297514537/811574542069137449) Discord channel.* --- ## Getting Started Running a Bee node means choosing a node type (full, light, or ultra-light), meeting the software, hardware, and network requirements, and picking an installation method. This guide covers each so you can choose the right setup. :::tip If you want to get a Bee node up and running ASAP, check out the [Quick Start](./quick-start.md) guide. ::: ## Overview This guide provides the essential background information to help you start running a Bee node, including: - [Types of Bee nodes and their features](./getting-started.md#node-types) - [Choosing the right node type](./getting-started.md#choosing-a-node-type) - [Software requirements](./getting-started.md#software-requirements) - [Hardware requirements](./getting-started.md#hardware-requirements) - [Network requirements](./getting-started.md#network-requirements) - [Installation methods](./getting-started.md#installation-methods) :::caution[New Bee Users: Read This Guide in Full] For new Bee users, it is strongly recommended to read through this ***entire guide page*** before proceeding with installation and setup. ::: ## Node Types Bee nodes can be run in three different modes, ***full***, ***light***, or ***ultra-light***. Full nodes provide complete access to all of Swarm's features including downloads, uploads, full participation in Swarm's incentives systems, and advanced messaging features such as PSS and GSOC. Light nodes are primarily for downloading and uploading only. Ultra-light nodes are the most limited, and only allow users to download a small amount of data with the free-tier limits set by full node operators. The [Node Types](./../working-with-bee/node-types.md) page provides you with an in-depth look into the features and limitations of each node type along with instructions for how to set node options for all three types. ## Choosing a Node Type The node type you need to run will differ depending on your use-case: | Use Case | Recommended Node Type(s) | Details | |------------------------------|-------------------------|---------| | **Basic Interaction with Swarm** | Ultra-Light, Light | Ultra-light nodes allow limited free-tier downloads. Light nodes support both uploads and downloads and run efficiently in the background. [Swarm Desktop](https://www.ethswarm.org/build/desktop) provides an easy way to set up either type. | | **DApp Development** | Light, Full | Light nodes are sufficient for many DApp use cases. Full nodes are required for advanced features like GSOC and PSS. | | **Earning xBZZ & Supporting the Network** | Full | Full nodes are necessary for storage incentives and long-term xBZZ earnings. Running multiple nodes? Consider using [Docker](https://www.docker.com/), [Docker Compose](https://docs.docker.com/compose/), or [Kubernetes](https://kubernetes.io/) for easier management. | Refer to the [Node Types](./../working-with-bee/node-types.md) page for deep dive into each node type, their features and limitations, and configuration instructions. ## Requirements ### Software Requirements #### Recommended Operating Systems - Officially supported systems are listed in the [Bee releases](https://github.com/ethersphere/bee/releases). - You can [build from source](./build-from-source.md) if your OS is unsupported. - **Swarm Desktop users** can use macOS, Windows, or Linux. - **Linux/macOS recommended**: Most tools and documentation are designed for Unix-based systems. - **Windows users**: While a Window release of Bee is available, you may also consider using [WSL](https://learn.microsoft.com/en-us/windows/wsl/install) and using a Linux version of Bee. #### Essential Tools While not strictly required, these tools will *greatly* simplify your experience working with Bee nodes: - **[`jq`](https://jqlang.org/)**: Formats JSON responses (recommended for API users). - **[`curl`](https://curl.se/)**: Used for sending API requests (required for API interactions). - **[Swarm CLI](./../working-with-bee/swarm-cli.md)**: Terminal-based Bee node management. - **[Bee JS](./../../develop/tools-and-features/bee-js.md)**: JavaScript library for programmatic API access. ### Hardware Requirements All three node types run on ordinary consumer hardware. None of them requires a powerful machine. #### Light and Ultra-Light Light and ultra-light nodes have very minimal CPU, RAM, disk and network requirements, and run on practically any commercially available computer hardware and internet connection. #### Full Node Full nodes have modest CPU and RAM requirements too, but they use more disk space and require a more sustained bandwidth than the lighter modes. They also need a Gnosis Chain RPC endpoint and some xDAI to cover gas fees, including the chequebook deployment transaction. See [full node specifications](./../working-with-bee/node-types.md#full-node-specifications) on the Node Types page for the complete list of hardware and funding requirements. Staking and receiving storage incentives may require more CPU power. Test node performance with [`/rchash`](https://docs.ethswarm.org/docs/bee/working-with-bee/bee-api/#rchash) before deciding to participate in the redistribution game. ### Network Requirements A reliable, high-speed internet connection is recommended when running a full node, while ultra-light and light nodes require less bandwidth. The actual amount of bandwidth consumption depends on the node type and use-case: - **Full Node**: High bandwidth usage due to constant chunk syncing, and even greater utilization if also used for uploads / downloads. - **Light Node**: Moderate usage, based on data transfer volume. - **Ultra-Light Node**: Minimal usage, bandwidth utilization restricted based on free-tier download limits. #### RPC Endpoint :::warning ***Free public RPC endpoints are discouraged*** since they may enforce rate limiting or may not store the historical smart contract data required by Bee nodes. [Read more](./../working-with-bee/configuration.md#setting-blockchain-rpc-endpoint). ::: An [RPC (Remote Procedure Call) endpoint](./../../references/glossary.md#rpc-endpoint) is required to allow your node to interact with **Gnosis Chain**, which is required for transactions like purchasing postage stamps, staking xBZZ, and storage incentives related transactions. Bee nodes use the **`--blockchain-rpc-endpoint`** configuration option to specify which Gnosis Chain RPC service to connect to. This can be: - A [self-hosted Gnosis Chain node](https://docs.gnosischain.com/node), giving full control over blockchain interactions but requiring additional setup and maintenance (recommended). - A **private and paid endpoint** from a third-party service provider. - A **public and free endpoint**, such as this free one from the Fair Data Society: `https://xdai.fairdatasociety.org` :::info A well-maintained list of both free and paid RPC endpoint providers can be found in the [Gnosis Chain documentation](https://docs.gnosischain.com/tools/RPC%20Providers/). ::: Without a properly configured RPC endpoint, a Bee node cannot interact with the blockchain, meaning it will be ***unable*** to: * Buy postage stamps * Stake tokens * Make blockchain transactions #### NAT and Port Forwarding If running Bee on a home network, there is a good chance it is behind NAT by default. Often simply [enabling port forwarding](https://www.noip.com/support/knowledgebase/general-port-forwarding-guide) will be enough to allow your node to start communicating smoothly with the rest of the network. See [this page](./connectivity.md) for more information on how to make sure your node can communicate with the network. For VPS/cloud-based setups, connectivity is typically unrestricted. If your home network happens to be using [CGNAT (Carrier-Grade NAT)](https://en.wikipedia.org/wiki/Carrier-grade_NAT), you may face significant difficulty with setting up your node so it can connect with the rest of the network. Contacting your IP provider may be required. ## Installation Methods ### [Swarm Desktop](./../../desktop/introduction.md) - Best for beginners. - GUI-based interface. ### [Shell Script Install](./shell-script.md) - Quick setup using a minimal script. - Requires manual configuration for background operation. ### [Docker Install](./docker.md) - Suitable for running multiple nodes. - Offers easy container management. ### [Package Manager Install](./package-manager.md) - Uses APT, RPM, or Homebrew. - Runs Bee as a background service. ### [Building from Source](./build-from-source.md) - Most flexible, but requires advanced setup. --- ## Hive Due to the mechanics of Swarm's [storage incentives](./../../concepts/incentives/redistribution-game.md), node operators may wish to run multiple nodes in order to maximize earning potential. Read [The Book of Swarm](https://www.ethswarm.org/the-book-of-swarm-2.pdf) for more information on how the swarm comes together. ## Docker Up-to-date [Docker images for Bee](./docker.md) are provided. ## Docker Compose Running multiple Bee nodes is easier with `docker-compose`. Check out the Docker compose section of the [Docker README](https://github.com/ethersphere/bee/tree/master/packaging/docker). ## Helm If you plan to run a large number of Bee nodes and you have experience using Kubernetes with Helm, you can have a look at how we manage our cluster under [Ethersphere/helm](https://github.com/ethersphere/helm/tree/master/charts/bee). ## Manual Setup If you just want to run a handful of Bee nodes, you can run multiple Bee nodes by creating separate configuration files. Create your first configuration file by running ```console bee printconfig &> bee-config-1.yaml ``` Make as many copies of bee-config-1.yaml as you want to run Bee nodes. Increment the number in the name (`bee-config-1` to `bee-config-2`) for each new configuration file. Configure your nodes as desired, but ensure that the values `api-addr`, `data-dir` and `p2p-addr` are unique for each configuration. ## Monitoring See the [logging section](./../working-with-bee/logs-and-files.md) for more information on how to access your node's metrics. Share your community creations (such as [swarmMonitor](https://github.com/doristeo/SwarmMonitoring) - thanks doristeo!) in the [#node-operators](https://discord.gg/kHRyMNpw7t) channel of our Discord server so we can add you to our list of all things that are [awesome](https://github.com/ethersphere/awesome-swarm) and Swarm. 🧡 --- ## Package Manager Install The Bee client can be [installed through a variety of package managers,](./package-manager.md) including [APT](https://en.wikipedia.org/wiki/APT_(software)), [RPM](https://en.wikipedia.org/wiki/RPM_Package_Manager), and [Homebrew](https://en.wikipedia.org/wiki/Homebrew_(package_manager)). :::caution When installed using a package manager, Bee is set up so it can be started to run as a background service using `systemctl` or `brew services` (depending on the package manager used). However, Bee node installed via a package manager can also be started using the standard `bee start` command. When a node is started using the `bee start` command the node process will be bound to the terminal session and will exit if the terminal is closed. Furthermore, depending on which of these startup methods was used, [*the default Bee directories will be different*](./../working-with-bee/configuration.md#default-data-and-config-directories). For each startup method, a different default data directory is used, so each startup method will essentially be spinning up a totally different node. ::: ## Install Bee Bee is available for Linux in .rpm and .deb package format for a variety of system architectures, and is available for MacOS through Homebrew. See the [releases](https://github.com/ethersphere/bee/releases) page of the Bee repo for all available packages. One of the advantages of this method is that it automatically configures Bee to run as a background service during installation. Get GPG key: ```bash curl -fsSL https://repo.ethswarm.org/apt/gpg.key | sudo gpg --dearmor -o /usr/share/keyrings/ethersphere-apt-keyring.gpg ``` Set up repo inside apt-get sources: ```bash echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ethersphere-apt-keyring.gpg] https://repo.ethswarm.org/apt \ * *" | sudo tee /etc/apt/sources.list.d/ethersphere.list > /dev/null ``` Install package: ```bash sudo apt-get update sudo apt-get install bee ``` Set up repo: ```bash echo "[ethersphere] name=Ethersphere Repo baseurl=https://repo.ethswarm.org/yum/ enabled=1 gpgcheck=0" | sudo tee /etc/yum.repos.d/ethersphere.repo ``` Install package: ```bash yum install bee ``` ```bash brew tap ethersphere/tap brew install swarm-bee ``` You should see the following output to your terminal after a successful install (your default 'Config' location will vary depending on your operating system): ```bash Reading package lists... Done Building dependency tree... Done Reading state information... Done The following NEW packages will be installed: bee 0 upgraded, 1 newly installed, 0 to remove and 37 not upgraded. Need to get 0 B/27.2 MB of archives. After this operation, 50.8 MB of additional disk space will be used. Selecting previously unselected package bee. (Reading database ... 82381 files and directories currently installed.) Preparing to unpack .../archives/bee_2.3.0_amd64.deb ... Unpacking bee (2.3.0) ... Setting up bee (2.3.0) ... Logs: journalctl -f -u bee.service Config: /etc/bee/bee.yaml Bee requires a Gnosis Chain RPC endpoint to function. By default this is expected to be found at ws://localhost:8546. Please see https://docs.ethswarm.org/docs/bee/installation/getting-started for more details on how to configure your node. After you finish configuration run 'sudo bee-get-addr' and fund your node with XDAI, and also XBZZ if so desired. Created symlink /etc/systemd/system/multi-user.target.wants/bee.service → /lib/systemd/system/bee.service. ``` ## Configure Bee When Bee is installed using a package manager, a `bee.yaml` file containing the default configuration will be generated. :::info While this package manager install guide uses the `bee.yaml` file for setting configuration options, there are [several other available methods for setting node options](./../working-with-bee/configuration.md). ::: After installation, you can check that the file was successfully generated and contains the [default configuration](https://github.com/ethersphere/bee/tree/master/packaging) for your system: ```bash test -f /etc/bee/bee.yaml && echo "bee.yaml exists." cat /etc/bee/bee.yaml ``` ```bash test -f /opt/homebrew/etc/swarm-bee/bee.yaml && echo "$FILE exists." cat /opt/homebrew/etc/swarm-bee/bee.yaml ``` ```bash test -f /usr/local/etc/swarm-bee/bee.yaml && echo "$FILE exists." cat /usr/local/etc/swarm-bee/bee.yaml ``` The configuration printed to the terminal should match the default configuration for your operating system. See the [the packaging section of the Bee repo](https://github.com/ethersphere/bee/tree/master/packaging) for the default configurations for a variety of systems. In particular, pay attention to the `config` and `data-dir` values, as these differ depending on your system. If your config file is missing you will need to create it yourself. Create the `bee.yaml` config file and save it with [the default configuration](https://github.com/ethersphere/bee/blob/master/packaging/bee.yaml). ```bash sudo touch /etc/bee/bee.yaml sudo vi /etc/bee/bee.yaml ``` Create the `bee.yaml` config file and save it with the [the default configuration](https://github.com/ethersphere/bee/blob/master/packaging/homebrew-arm64/bee.yaml). ```bash sudo touch /opt/homebrew/etc/swarm-bee/bee.yaml sudo sudo vi /opt/homebrew/etc/swarm-bee/bee.yaml ``` Create the `bee.yaml` config file and save it with the [the default configuration](https://github.com/ethersphere/bee/blob/master/packaging/homebrew-amd64/bee.yaml). ```bash sudo touch /usr/local/etc/swarm-bee/bee.yaml sudo vi /usr/local/etc/swarm-bee/bee.yaml ``` ### Set Node Type See the [Getting Started guide](./getting-started.md#choosing-a-node-type) if you're not sure which type of node to run. Once you've decided which node type is appropriate for you, refer to the [configuration section](./../working-with-bee/configuration.md#node-types) for instructions on setting the options for your preferred node type. ### Set Target Neighborhood When installing your Bee node it will automatically be assigned a neighborhood. However, when running a full node with staking there are benefits to periodically updating your node's neighborhood. Learn more about why and how to set your node's target neighborhood [here](./set-target-neighborhood.md). ## Start Node Use the appropriate command for your system to start your node: ```bash sudo systemctl start bee ``` ```bash brew services start swarm-bee ``` ```bash Welcome to Swarm.... Bzzz Bzzzz Bzzzz \ / \ o ^ o / \ ( ) / ____________(%%%%%%%)____________ ( / / )%%%%%%%( \ \ ) (___/___/__/ \__\___\___) ( / /(%%%%%%%)\ \ ) (__/___/ (%%%%%%%) \___\__) /( )\ / (%%%%%) \ (%%%) ! DISCLAIMER: This software is provided to you "as is", use at your own risk and without warranties of any kind. It is your responsibility to read and understand how Swarm works and the implications of running this software. The usage of Bee involves various risks, including, but not limited to: damage to hardware or loss of funds associated with the Ethereum account connected to your node. No developers or entity involved will be liable for any claims and damages associated with your use, inability to use, or your interaction with other nodes or the software. version: 2.2.0-06a0aca7 - planned to be supported until 11 December 2024, please follow https://ethswarm.org/ "time"="2024-09-24 18:15:34.383102" "level"="info" "logger"="node" "msg"="bee version" "version"="2.2.0-06a0aca7" "time"="2024-09-24 18:15:34.428546" "level"="info" "logger"="node" "msg"="swarm public key" "public_key"="0373fe2ab33ab836635fc35864cf708fa0f4a775c0cf76ca851551e7787b58d040" "time"="2024-09-24 18:15:34.520686" "level"="info" "logger"="node" "msg"="pss public key" "public_key"="03a341032724f1f9bb04f1d9b22607db485cccd74174331c701f3a6957d94d95c1" "time"="2024-09-24 18:15:34.520716" "level"="info" "logger"="node" "msg"="using ethereum address" "address"="0x1A801dd3ec955E905ca424a85C3423599bfb0E66" "time"="2024-09-24 18:15:34.533789" "level"="info" "logger"="node" "msg"="fetching target neighborhood from suggester" "url"="https://api.swarmscan.io/v1/network/neighborhoods/suggestion" "time"="2024-09-24 18:15:36.773501" "level"="info" "logger"="node" "msg"="mining a new overlay address to target the selected neighborhood" "target"="00100010110" "time"="2024-09-24 18:15:36.776550" "level"="info" "logger"="node" "msg"="using overlay address" "address"="22d502d022de0f8e9d477bc61144d0d842d9d82b8241568c6fe4e41f0b466615" "time"="2024-09-24 18:15:36.776576" "level"="info" "logger"="node" "msg"="starting with an enabled chain backend" "time"="2024-09-24 18:15:37.388997" "level"="info" "logger"="node" "msg"="connected to blockchain backend" "version"="erigon/2.60.7/linux-amd64/go1.21.5" "time"="2024-09-24 18:15:37.577840" "level"="info" "logger"="node" "msg"="using chain with network network" "chain_id"=100 "network_id"=1 "time"="2024-09-24 18:15:37.593747" "level"="info" "logger"="node" "msg"="starting debug & api server" "address"="127.0.0.1:1633" "time"="2024-09-24 18:15:37.969782" "level"="info" "logger"="node" "msg"="using default factory address" "chain_id"=100 "factory_address"="0xC2d5A532cf69AA9A1378737D8ccDEF884B6E7420" "time"="2024-09-24 18:15:38.160249" "level"="info" "logger"="node/chequebook" "msg"="no chequebook found, deploying new one." "time"="2024-09-24 18:15:38.728534" "level"="warning" "logger"="node/chequebook" "msg"="cannot continue until there is at least min xDAI (for Gas) available on address" "min_amount"="0.0003750000017" "address"="0x1A801dd3ec955E905ca424a85C3423599bfb0E66" ``` Take note of the lines: ```bash "time"="2024-09-24 18:15:34.520716" "level"="info" "logger"="node" "msg"="using ethereum address" "address"="0x1A801dd3ec955E905ca424a85C3423599bfb0E66" ``` and ```bash "time"="2024-09-24 18:15:38.728534" "level"="warning" "logger"="node/chequebook" "msg"="cannot continue until there is at least min xDAI (for Gas) available on address" "min_amount"="0.0003750000017" "address"="0x1A801dd3ec955E905ca424a85C3423599bfb0E66" ``` The address referred to in both of these lines is your node's Gnosis Chain address. The second one indicates that the address does not have enough xDAI in order to deploy your node's chequebook contract which is used to pay for bandwidth incentives. You will see this warning if you have configured your node to run as a `full` or `light` node, but it should be absent for `ultra-light` nodes. ## Fund Node Depending on your chosen node type, (full, light, or ultra-light), you will want to fund your node with differing amounts of xBZZ and xDAI. See [this section](./fund-your-node.md) for more information on how to fund your node. ### Restart and Wait for Initialisation After funding your node, use the appropriate command for your system below and wait for it to initialize: ```bash sudo systemctl start bee ``` ```bash brew services start swarm-bee ``` When first started in full or light mode, Bee must deploy a chequebook to the Gnosis Chain blockchain, and sync the postage stamp batch store so that it can check chunks for validity when storing or forwarding them. This can take a while, so please be patient! Once this is complete, you will see Bee starting to add peers and connect to the network. You can keep an eye on progress by watching the logs while this is taking place. ```bash sudo journalctl --lines=100 --follow --unit bee ``` ```bash tail -f /opt/homebrew/var/log/swarm-bee/bee.log ``` ```bash tail -f /usr/local/var/log/swarm-bee/bee.log ``` *If you've started your node with `bee start`, simply observe the logs printed to your terminal.* If all goes well, you will see your node automatically begin to connect to other Bee nodes all over the world. ``` INFO[2020-08-29T11:55:16Z] greeting from peer: b6ae5b22d4dc93ce5ee46a9799ef5975d436eb63a4b085bfc104fcdcbda3b82c ``` Now your node will begin to request chunks of data that fall within your *radius of responsibilty*—data that you will then serve to other p2p clients running in the swarm. Your node will then begin to respond to requests for these chunks from other peers. :::tip Incentivisation In Swarm, storing, serving and forwarding chunks of data to other nodes can earn you rewards! Follow [this guide](./../working-with-bee/cashing-out.md) to learn how to regularly cash out cheques other nodes send you in return for your services so that you can get your xBZZ! ::: Your Bee client has now generated an elliptic curve key pair similar to an Ethereum wallet. These are stored in your [data directory](./../working-with-bee/configuration.md), in the `keys` folder. :::danger Keep Your Keys and Password Safe! Your keys and password are very important, back up these files and store them in a secure place that only you have access to. With great privacy comes great responsibility - while no-one will ever be able to guess your key - you will not be able to recover them if you lose them either, so be sure to look after them well and [keep secure backups](/docs/bee/working-with-bee/backups). ::: ## Check if Bee is Working First check that the correct version of Bee is installed: ```bash bee version ``` ``` 2.3.0 ``` Once the Bee node has been funded, the chequebook deployed, and postage stamp batch store synced, its HTTP [API](./../working-with-bee/bee-api.md) will start listening at `localhost:1633` (for `full` or `light` nodes - for an `ultra-light` node, it should be initialized and the API should be available more rapidly). To check everything is working as expected, send a GET request to localhost port 1633. ```bash curl localhost:1633 ``` ``` Ethereum Swarm Bee ``` Success! The Bee API is now listening! Next, let's see if we have connected with any peers by sending a query to the Bee API (port 1633 by default - `localhost:1633`). :::info Here we are using the `jq` [utility](https://jqlang.org/) to parse our javascript. Use your package manager to install `jq`, or simply remove everything after and including the first `|` to view the raw json without it. ::: ```bash curl -s localhost:1633/peers | jq ".peers | length" ``` ``` 87 ``` Perfect! We are accumulating peers, this means you are connected to the network, and ready to start [using Bee](/docs/develop/introduction) to [upload and download](/docs/develop/upload-and-download) content or host and browse [websites](./../../develop/host-your-website.md) hosted on the Swarm network. Welcome to the swarm! 🐝 🐝 🐝 🐝 🐝 ## Back Up Keys Once your node is up and running, make sure to [back up your keys](./../working-with-bee/backups.md). ## Deposit Stake (Optional) While depositing stake is not required to run a Bee node, it is required in order for a node to receive rewards for sharing storage with the network. You will need to [deposit xBZZ to the staking contract](./../working-with-bee/staking.md) for your node. To do this, send a minimum of 10 xBZZ to your nodes' wallet and run: ```bash curl -X POST localhost:1633/stake/100000000000000000 ``` This will initiate a transaction on-chain which deposits the specified amount of xBZZ into the staking contract. Storage incentive rewards are only available for full nodes which are providing storage capacity to the network. *Note that SWAP rewards (bandwidth incentives paid for forwarding chunks) are available to all **full** nodes, regardless of whether or not they stake xBZZ in order to participate in the storage incentives system.* ## Next Steps to Consider ### Access the Swarm If you'd like to start uploading or downloading files to Swarm, [start here](./../../develop/introduction.md). ### Explore the API The [Bee API](./../working-with-bee/bee-api.md) is the primary method for interacting with Bee and getting information about Bee. After installing Bee and getting it up and running, it's a good idea to start getting familiar with the API. ### Run a hive! If you would like to run a hive of many Bees, check out the [hive operators](./hive.md) section for information on how to operate and monitor many Bees at once. ### Start building DAPPs on Swarm If you would like to start building decentralised applications on Swarm, check out our section for [developing with Bee](./../../develop/introduction.md). --- ## Quickstart This guide will help you install and run a Bee [light node](./../working-with-bee/node-types.md) using the [shell script](./shell-script.md) install method. After explaining how to install and start the node, the guide then explains how to use the [`swarm-cli` command line tool](./../working-with-bee/swarm-cli.md) to find your node's address, fund your node, and fully initialize it so that it is ready interact with the network. :::tip A "light" node can download and upload data from Swarm but does not share its disk space with the network and does not earn rewards. [Learn more](./../working-with-bee/node-types.md). ::: ## Requirements - **Linux or macOS** (The shell script installation method does **not** support Windows natively. Windows users can use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install).) - [Node.js (v18 or higher)](https://nodejs.org/) - [npm (Node Package Manager)](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) - [`curl`](https://curl.se/) or [`wget`](https://www.gnu.org/software/wget/) (Check with `curl --version` or `wget --version`) - [~0.20 xBZZ on Gnosis Chain](./fund-your-node.md#how-to-get-xbzz) - [~0.01 xDAI on Gnosis Chain](./fund-your-node.md#how-to-get-xdai) :::info Although `BZZ` is the official symbol of the token on both Ethereum and Gnosis Chain, the term `xBZZ` is widely used by the Swarm community and in documentation to indicate that it is BZZ on Gnosis Chain (not Ethereum). ::: ## Install Bee Run the shell script using `curl` or `wget`: :::tip We specify `TAG=v2.8.1` to indicate which Bee version to install. You can find available versions in the ["releases" section](https://github.com/ethersphere/bee/releases) of the Bee GitHub repo. ::: ```bash curl -s https://raw.githubusercontent.com/ethersphere/bee/master/install.sh | TAG=v2.8.1 bash ``` OR ```bash wget -q -O - https://raw.githubusercontent.com/ethersphere/bee/master/install.sh | TAG=v2.8.1 bash ``` Verify installation: ```bash bee version ``` ## Install Swarm-CLI Requires **Node.js 18+**. Install using **npm**: ```bash npm install --global @ethersphere/swarm-cli ``` Verify installation: ```bash swarm-cli --version ``` ## Start Your Bee Node Start Bee with a secure password: ```bash bee start \ --password YOUR_SECURE_PASSWORD \ --verbosity 5 \ --swap-enable \ --api-addr 127.0.0.1:1633 \ --blockchain-rpc-endpoint https://xdai.fairdatasociety.org ``` Example output: ```bash Welcome to Swarm.... Bzzz Bzzzz Bzzzz "Share the knowledge" - in memory of ldeffenb \ / \ o ^ o / \ ( ) / ____________(%%%%%%%)____________ ( / / )%%%%%%%( \ \ ) (___/___/__/ \__\___\___) ( / /(%%%%%%%)\ \ ) (__/___/ (%%%%%%%) \___\__) /( )\ / (%%%%%) \ (%%%) ! DISCLAIMER: This software is provided to you "as is", use at your own risk and without warranties of any kind. It is your responsibility to read and understand how Swarm works and the implications of running this software. The usage of Bee involves various risks, including, but not limited to: damage to hardware or loss of funds associated with the Ethereum account connected to your node. No developers or entity involved will be liable for any claims and damages associated with your use, inability to use, or your interaction with other nodes or the software. "time"="2026-07-07 16:52:59.641444" "level"="info" "logger"="node" "msg"="bee version" "version"="2.8.1-7cf53193" "time"="2026-07-07 16:52:59.793257" "level"="info" "logger"="node" "msg"="swarm public key" "public_key"="02d8d7e1ca6b3b43653ae27e35a375dd74e3ce2f40587fd264bc7268ed918650ab" "time"="2026-07-07 16:53:00.087534" "level"="info" "logger"="node" "msg"="pss public key" "public_key"="02aaae4ede42f47f48aa5182df4b94039ca71254f44ebc5383d5a67f71fe7e6156" "time"="2025-03-04 11:13:10.268479" "level"="info" "logger"="node" "msg"="using ethereum address" "address"="0x003842B26B3dB292Cf84d5969E71c0d1e93F5578" "time"="2025-03-04 11:13:10.288418" "level"="info" "logger"="node" "msg"="using overlay address" "address"="fe38346dd89e4211c0e60195ee73e38d2c2ee2fe2b914b771d4ad503cfedbd3c" "time"="2025-03-04 11:13:10.288474" "level"="info" "logger"="node" "msg"="starting with an enabled chain backend" "time"="2025-03-04 11:13:10.987200" "level"="info" "logger"="node" "msg"="connected to blockchain backend" "version"="Nethermind/v1.30.3+87c86379/linux-x64/dotnet9.0.0" "time"="2025-03-04 11:13:11.196067" "level"="info" "logger"="node" "msg"="using chain with network network" "chain_id"=100 "network_id"=1 "time"="2025-03-04 11:13:11.211976" "level"="info" "logger"="node" "msg"="starting debug & api server" "address"="127.0.0.1:1633" "time"="2025-03-04 11:13:11.623998" "level"="info" "logger"="node" "msg"="using default factory address" "chain_id"=100 "factory_address"="0xC2d5A532cf69AA9A1378737D8ccDEF884B6E7420" "time"="2025-03-04 11:13:11.675186" "level"="info" "logger"="node/chequebook" "msg"="no chequebook found, deploying new one." "time"="2025-03-04 11:13:11.723451" "level"="warning" "logger"="node/chequebook" "msg"="cannot continue until there is at least min xDAI (for Gas) available on address" "min_amount"="0.000250000002" "address"="0x003842B26B3dB292Cf84d5969E71c0d1e93F5578" ``` 🎉 Congratulations! You've just successfully installed and started your first Bee node 🐝! ## Get Your Node’s Address The final line of the logs in the previous step lets us know that we need to fund our node to continue, and shows our node's Gnosis chain address. Copy the address and save it for the next step: ```bash "time"="2025-03-04 11:13:11.723451" "level"="warning" "logger"="node/chequebook" "msg"="cannot continue until there is at least min xDAI (for Gas) available on address" "min_amount"="0.000250000002" "address"="0x003842B26B3dB292Cf84d5969E71c0d1e93F5578" ``` You can also view your node's addresses any time using the `swarm-cli addresses` command: ```bash swarm-cli addresses ``` Example output: ```bash Node Addresses ----------------------------------------------------------------------------------------------------------------------------------- Ethereum: 0x003842b26b3db292cf84d5969e71c0d1e93f5578 Overlay: fe38346dd89e4211c0e60195ee73e38d2c2ee2fe2b914b771d4ad503cfedbd3c PSS Public Key: 03a3166e04b749ab3d04fda8a41180598ff2eed01a8096fb72d2c7da393a47c46a Public Key: 02b19880b8d024eac3bf8afa3fa85b31b72fcfd491cebc6af78ddd85ff97f65416 Underlay: /ip4/127.0.0.1/tcp/1634/p2p/QmPbXzjN9mzYnpxsMn6ftFvvUuf4VArcmR6oGtpf1mRgWt /ip4/172.25.128.69/tcp/1634/p2p/QmPbXzjN9mzYnpxsMn6ftFvvUuf4VArcmR6oGtpf1mRgWt /ip6/::1/tcp/1634/p2p/QmPbXzjN9mzYnpxsMn6ftFvvUuf4VArcmR6oGtpf1mRgWt ``` ## Fund Your Node Send **xDAI** (to pay for transaction fees on Gnosis Chain) and **xBZZ** (for uploads and staking) to your node’s Ethereum address on **Gnosis Chain**. - **xDAI:** 0.01 xDAI is enough to start a light node - **xBZZ:** 0.20 xBZZ is enough to upload a small amount of data Learn how to [get xDAI and xBZZ](./fund-your-node.md#getting-tokens) if you need some. :::tip If you wait too long to fund your node it may shut itself down. In that case, simply use the same startup command to start the node again. ::: ## Wait to Sync (~5 Minutes) After starting and funding a Bee light node for the first time, the node will automatically issue a [transaction](https://gnosisscan.io/tx/0xf8048c4e8020ccef842c9a901e6262e9c06d6f5926ff31bdb7dd9d7274dcf19c) on Gnosis Chain to deploy the node's [chequebook contract](./../../concepts/incentives/bandwidth-incentives.md#chequebook-contract). The node then needs to sync blockchain data before it can buy a postage stamp batch. The process may take **~5 minutes** depending on your RPC provider and network speed. You can check your node's syncing progress with the `swarm-cli status` command: ```bash swarm-cli status ``` ```bash Bee API: http://localhost:1633 [OK] Version: 2.8.1-7cf53193 Mode: light Chainsync Block: 39,566,742 / 41,710,807 (Δ 2,144,065) Topology ERROR Request failed with status code 503 There may be additional information in the Bee logs. ``` The `Chainsync` section tells us how many blocks our node has synced so far out of the total Gnosis Chain blocks (and the number after the Δ symbol shows how many blocks still need to be synced): ```bash Chainsync Block: 33,515,656 / 38,855,407 (Δ 5,339,751) ``` The `Topology` section will show information about which other nodes your own node is connected with. It will display an `ERROR` until the node is fully initialized. After several minutes, your node will be fully synced, and can now interact with the Swarm network - we can use `swarm-cli status` again to confirm: ```bash swarm-cli status ``` ```bash Bee API: http://localhost:1633 [OK] Version: 2.8.1-7cf53193 Mode: light Chainsync Block: 41,711,260 / 41,711,268 (Δ 8) Topology Connected Peers: 156 Population: 2693 Depth: 10 Wallet xBZZ: 0.0000000000000000 xDAI: 0.009787142484816165 Chequebook Available xBZZ: 0.0000000000000000 Total xBZZ: 0.0000000000000000 ``` ## Next Steps With your node now fully synced, you're ready start start learning how to [develop on Swarm](./../../develop/introduction.md). --- ## Set Target Neighborhood In older versions of Bee, [neighborhood](./../../concepts/DISC/neighborhoods.md) assignment was random by default. However, we can maximize a node's chances of winning xBZZ and also strengthen the resiliency of the network by strategically assigning neighborhoods to new nodes (see the [staking section](./../working-with-bee/staking.md) for more details). Therefore the default Bee configuration now includes the `neighborhood-suggester` option, which is set by default to use the Swarmscan neighborhood suggester (`https://api.swarmscan.io/v1/network/neighborhoods/suggestion`). You can use an alternative suggester URL, but it must return a JSON response in the following format: `{"neighborhood":"101000110101"}`. However, we currently recommend using only the default suggester. :::info The Swarmscan neighborhood selector prioritizes the least populated neighborhood. If a neighborhood contains imbalanced sub-neighborhoods, it will suggest the least populated sub-neighborhood instead. Furthermore, the suggester will temporarily de-prioritize previously suggested neighborhoods based on the assumption that a new node is being created in each suggested neighborhood so that multiple nodes do not simultaneously attempt to join the same neighborhood. ::: ## Setting Neighborhood Manually It's recommended to use the default `neighborhood-suggester` configuration for choosing your node's neighborhood, however you may also set your node's neighborhood manually using the `target-neighborhood` option. To use this option, it's first necessary to identify potential target neighborhoods. You can find underpopulated neighborhoods using the [Swarmscan website](https://swarmscan.io/neighborhoods). It ranks neighborhoods from least to most populated and displays their leading binary bits. Simply copy the leading bits from one of the least populated neighborhoods (for example, `0010100001`) and use it to set `target-neighborhood`. After doing so, an overlay address within that neighborhood will be generated when starting Bee for the first time. ```yaml # bee.yaml target-neighborhood: "0010100001" ``` You can also use the [Swarmscan API endpoint](https://api.swarmscan.io/#tag/Network/paths/~1v1~1network~1neighborhoods~1suggestion/get) to programmatically retrieve a suggested neighborhood: ```bash curl https://api.swarmscan.io/v1/network/neighborhoods/suggestion ``` A suggested neighborhood will be returned: ```bash {"neighborhood":"1111110101"} ``` --- ## Shell Script Install The official [shell script](https://github.com/ethersphere/bee/blob/master/install.sh) provided by Swarm automatically detects your system and installs the correct version of Bee. This installation method is an excellent choice if you're looking for a minimalistic and flexible option for your Bee node installation. :::warning Note that we append 127.0.0.1 (localhost) to our Bee API's port (1633 by default), since we do not want to expose our Bee API endpoint to the public internet, as that would allow anyone to control our node. Make sure you do the same. Additionally, it's recommended to use a firewall to restrict access to your node(s). ::: :::info This guide uses command line flag options in the node startup commands such as `--blockchain-rpc-endpoint`, however, there are [several other methods available for configuring options](./../working-with-bee/configuration.md). ::: ## Install and Start Your Node Below is a step-by-step guide for installing and setting up your Bee node using the shell script installation method. ### Run Shell Script Run the install shell script using either `curl` or `wget`: :::caution In the example below, the version is specified using `TAG=v2.8.1`. Check the [latest Bee releases](https://github.com/ethersphere/bee/tags) and if needed, update the command to install the most recent version (note that in tags containing "rc," the abbreviation stands for "release candidate", and these versions should be used for testing purposes only). ::: :::info Note that while this shell script supports many commonly used Unix-like systems, it is not quite a universal installer tool. The architectures it supports include: **1. Linux:** - `linux-386` (32-bit x86) - `linux-amd64` (64-bit x86) - `linux-arm64` (64-bit ARM) - `linux-armv6` (32-bit ARM v6) **2. macOS (Darwin):** - `darwin-arm64` (Apple Silicon, M1/M2/M3) - `darwin-amd64` (Intel-based Mac) This means the script works on most modern Linux distributions and macOS versions that match these architectures. Windows users can use [WSL](https://learn.microsoft.com/en-us/windows/wsl/install). ::: :::caution You may need to install [`curl`](https://curl.se/) or [`wget`](https://www.gnu.org/software/wget/) if your system doesn't have one of them pre-installed and the shell script command fails to run. ::: ```bash curl -s https://raw.githubusercontent.com/ethersphere/bee/master/install.sh | TAG=v2.8.1 bash ``` **wget** ```bash wget -q -O - https://raw.githubusercontent.com/ethersphere/bee/master/install.sh | TAG=v2.8.1 bash ``` Let's check that the script ran properly: ```bash= bee ``` If the script ran without any problems you should see this: ```bash= Ethereum Swarm Bee Usage: bee [command] Available Commands: start Start a Swarm node init Initialise a Swarm node deploy Deploy and fund the chequebook contract version Print version number db Perform basic DB related operations split Split a file into chunks printconfig Print default or provided configuration in yaml format help Help about any command completion Generate the autocompletion script for the specified shell Flags: --config string config file (default is $HOME/.bee.yaml) -h, --help help for bee Use "bee [command] --help" for more information about a command. ``` ### Node Startup Commands Let's try starting up our node for the first time with the command below. Make sure to pick a [strong password](https://xkcd.com/936/) of your own: Below are startup commands configured for each of the three Bee node types, ***full***, ***light***, and ***ultra-light***. Refer to the [Node Types](./../working-with-bee/node-types.md) page to learn more about each node type and decide which one best suits your needs. For the full node, we have `--full-node` and `--swap-enable` both enabled, and we've used `--blockchain-rpc-endpoint` to set our RPC endpoint as `https://xdai.fairdatasociety.org`. Your RPC endpoint may differ depending on your setup. ```bash bee start \ --password flummoxedgranitecarrot \ --full-node \ --swap-enable \ --api-addr 127.0.0.1:1633 \ --blockchain-rpc-endpoint https://xdai.fairdatasociety.org ``` For the light node, we omit `--full-node`, keeping the rest the same as the full node setup. ```bash bee start \ --password flummoxedgranitecarrot \ --swap-enable \ --api-addr 127.0.0.1:1633 \ --blockchain-rpc-endpoint https://xdai.fairdatasociety.org ``` For the ultra-light node, we omit all three of the relevant settings to disable them (since they default to `false`), `--full-node`, `--swap-enable`, and `--blockchain-rpc-endpoint`. ```bash bee start \ --password flummoxedgranitecarrot \ --api-addr 127.0.0.1:1633 ``` :::info Command explained: 1. **`bee start`**: This is the command to start the Bee node. 2. **`--password flummoxedgranitecarrot`**: The password to decrypt the private key associated with the node. Replace "flummoxedgranitecarrot" with your actual password. 3. **`--full-node`**: This option enables the node to run in full mode, sharing its disk with the network, and becoming eligible for staking. 4. **`--swap-enable`**: This flag enables SWAP, which is the bandwidth incentives scheme for Swarm. It will initiate a transaction to set up the SWAP chequebook on Gnosis Chain (required for light and full nodes). 5. **`--api-addr 127.0.0.1:1633`**: Specifies that the Bee API will be accessible locally only via `127.0.0.1` on port `1633` and not accessible to the public. 6. **`--blockchain-rpc-endpoint https://xdai.fairdatasociety.org`**: Sets the RPC endpoint for interacting with the Gnosis blockchain (required for light and full nodes). ::: ### Example Startup Output The node has successfully started, but it still needs funding with xDAI (for Gnosis Chain transactions) and xBZZ (for uploads, downloads, and staking). ```bash Welcome to Swarm.... Bzzz Bzzzz Bzzzz \ / \ o ^ o / \ ( ) / ____________(%%%%%%%)____________ ( / / )%%%%%%%( \ \ ) (___/___/__/ \__\___\___) ( / /(%%%%%%%)\ \ ) (__/___/ (%%%%%%%) \___\__) /( )\ / (%%%%%) \ (%%%) ! DISCLAIMER: This software is provided to you "as is", use at your own risk and without warranties of any kind. It is your responsibility to read and understand how Swarm works and the implications of running this software. The usage of Bee involves various risks, including, but not limited to: damage to hardware or loss of funds associated with the Ethereum account connected to your node. No developers or entity involved will be liable for any claims and damages associated with your use, inability to use, or your interaction with other nodes or the software. version: 2.2.0-06a0aca7 - planned to be supported until 11 December 2024, please follow https://ethswarm.org/ "time"="2024-09-24 18:15:34.383102" "level"="info" "logger"="node" "msg"="bee version" "version"="2.2.0-06a0aca7" "time"="2024-09-24 18:15:34.428546" "level"="info" "logger"="node" "msg"="swarm public key" "public_key"="0373fe2ab33ab836635fc35864cf708fa0f4a775c0cf76ca851551e7787b58d040" "time"="2024-09-24 18:15:34.520686" "level"="info" "logger"="node" "msg"="pss public key" "public_key"="03a341032724f1f9bb04f1d9b22607db485cccd74174331c701f3a6957d94d95c1" "time"="2024-09-24 18:15:34.520716" "level"="info" "logger"="node" "msg"="using ethereum address" "address"="0x1A801dd3ec955E905ca424a85C3423599bfb0E66" "time"="2024-09-24 18:15:34.533789" "level"="info" "logger"="node" "msg"="fetching target neighborhood from suggester" "url"="https://api.swarmscan.io/v1/network/neighborhoods/suggestion" "time"="2024-09-24 18:15:36.773501" "level"="info" "logger"="node" "msg"="mining a new overlay address to target the selected neighborhood" "target"="00100010110" "time"="2024-09-24 18:15:36.776550" "level"="info" "logger"="node" "msg"="using overlay address" "address"="22d502d022de0f8e9d477bc61144d0d842d9d82b8241568c6fe4e41f0b466615" "time"="2024-09-24 18:15:36.776576" "level"="info" "logger"="node" "msg"="starting with an enabled chain backend" "time"="2024-09-24 18:15:37.388997" "level"="info" "logger"="node" "msg"="connected to blockchain backend" "version"="erigon/2.60.7/linux-amd64/go1.21.5" "time"="2024-09-24 18:15:37.577840" "level"="info" "logger"="node" "msg"="using chain with network network" "chain_id"=100 "network_id"=1 "time"="2024-09-24 18:15:37.593747" "level"="info" "logger"="node" "msg"="starting debug & api server" "address"="127.0.0.1:1633" "time"="2024-09-24 18:15:37.969782" "level"="info" "logger"="node" "msg"="using default factory address" "chain_id"=100 "factory_address"="0xC2d5A532cf69AA9A1378737D8ccDEF884B6E7420" "time"="2024-09-24 18:15:38.160249" "level"="info" "logger"="node/chequebook" "msg"="no chequebook found, deploying new one." "time"="2024-09-24 18:15:38.728534" "level"="warning" "logger"="node/chequebook" "msg"="cannot continue until there is at least min xDAI (for Gas) available on address" "min_amount"="0.0003750000017" "address"="0x1A801dd3ec955E905ca424a85C3423599bfb0E66" ``` Here you can see that the node has started up successfully, but our node still needs to be funded with xDAI and xBZZ (xDAI for Gnosis Chain transactions and xBZZ for uploads/downloads). Continue to the next section for funding instructions. ```bash Welcome to Swarm.... Bzzz Bzzzz Bzzzz \ / \ o ^ o / \ ( ) / ____________(%%%%%%%)____________ ( / / )%%%%%%%( \ \ ) (___/___/__/ \__\___\___) ( / /(%%%%%%%)\ \ ) (__/___/ (%%%%%%%) \___\__) /( )\ / (%%%%%) \ (%%%) ! DISCLAIMER: This software is provided to you "as is", use at your own risk and without warranties of any kind. It is your responsibility to read and understand how Swarm works and the implications of running this software. The usage of Bee involves various risks, including, but not limited to: damage to hardware or loss of funds associated with the Ethereum account connected to your node. No developers or entity involved will be liable for any claims and damages associated with your use, inability to use, or your interaction with other nodes or the software. version: 2.2.0-06a0aca7 - planned to be supported until 11 December 2024, please follow https://ethswarm.org/ "time"="2025-01-24 12:57:21.274657" "level"="info" "logger"="node" "msg"="bee version" "version"="2.2.0-06a0aca7" "time"="2025-01-24 12:57:21.274854" "level"="warning" "logger"="node" "msg"="your node is outdated, please check for the latest version" "time"="2025-01-24 12:57:21.449064" "level"="info" "logger"="node" "msg"="swarm public key" "public_key"="03c356839a5570c758e812d0c248b135f0dc8ffa2b8404a97597e456f4fe5f7ee8" "time"="2025-01-24 12:57:21.805033" "level"="info" "logger"="node" "msg"="pss public key" "public_key"="036c63b7c544ad401a5dbfb463f71cda265eec74c1d0d9cbc9db2abd6b3e4f11e9" "time"="2025-01-24 12:57:21.805124" "level"="info" "logger"="node" "msg"="using ethereum address" "address"="0x5c39545873Bd663b0bB0716ED87dE0E399Aae419" "time"="2025-01-24 12:57:21.815765" "level"="info" "logger"="node" "msg"="using overlay address" "address"="74539eab1dbd5c722bb8ba10cef55f715e38f298b706fb1866af49f4fd15d8d3" "time"="2025-01-24 12:57:21.815855" "level"="info" "logger"="node" "msg"="starting with an enabled chain backend" "time"="2025-01-24 12:57:21.861341" "level"="info" "logger"="node" "msg"="connected to blockchain backend" "version"="Nethermind/v1.30.1+2b75a75a/linux-x64/dotnet9.0.0" "time"="2025-01-24 12:57:21.869117" "level"="info" "logger"="node" "msg"="using chain with network network" "chain_id"=100 "network_id"=1 "time"="2025-01-24 12:57:21.880930" "level"="info" "logger"="node" "msg"="starting debug & api server" "address"="127.0.0.1:1633" "time"="2025-01-24 12:57:21.897675" "level"="info" "logger"="node" "msg"="using default factory address" "chain_id"=100 "factory_address"="0xC2d5A532cf69AA9A1378737D8ccDEF884B6E7420" "time"="2025-01-24 12:57:21.911463" "level"="info" "logger"="node/chequebook" "msg"="no chequebook found, deploying new one." "time"="2025-01-24 12:57:21.938038" "level"="warning" "logger"="node/chequebook" "msg"="cannot continue until there is at least min xDAI (for Gas) available on address" "min_amount"="0.000250000002" "address"="0x5c39545873Bd663b0bB0716ED87dE0E399Aae419" ``` If you've started in ultra-light mode, you should see output which looks something like this, and you're done! Your node is now successfully running in ultra-light mode. You can now skip down to the final section on this page about logs and monitoring. ```bash root@noah-bee:~# bee start \ --password flummoxedgranitecarrot \ --api-addr 127.0.0.1:1633 Welcome to Swarm.... Bzzz Bzzzz Bzzzz \ / \ o ^ o / \ ( ) / ____________(%%%%%%%)____________ ( / / )%%%%%%%( \ \ ) (___/___/__/ \__\___\___) ( / /(%%%%%%%)\ \ ) (__/___/ (%%%%%%%) \___\__) /( )\ / (%%%%%) \ (%%%) ! DISCLAIMER: This software is provided to you "as is", use at your own risk and without warranties of any kind. It is your responsibility to read and understand how Swarm works and the implications of running this software. The usage of Bee involves various risks, including, but not limited to: damage to hardware or loss of funds associated with the Ethereum account connected to your node. No developers or entity involved will be liable for any claims and damages associated with your use, inability to use, or your interaction with other nodes or the software. version: 2.2.0-06a0aca7 - planned to be supported until 11 December 2024, please follow https://ethswarm.org/ "time"="2025-01-24 12:51:06.981505" "level"="info" "logger"="node" "msg"="bee version" "version"="2.2.0-06a0aca7" "time"="2025-01-24 12:51:06.981658" "level"="warning" "logger"="node" "msg"="your node is outdated, please check for the latest version" "time"="2025-01-24 12:51:07.131555" "level"="info" "logger"="node" "msg"="swarm public key" "public_key"="03c356839a5570c758e812d0c248b135f0dc8ffa2b8404a97597e456f4fe5f7ee8" "time"="2025-01-24 12:51:07.402847" "level"="info" "logger"="node" "msg"="pss public key" "public_key"="036c63b7c544ad401a5dbfb463f71cda265eec74c1d0d9cbc9db2abd6b3e4f11e9" "time"="2025-01-24 12:51:07.402915" "level"="info" "logger"="node" "msg"="using ethereum address" "address"="0x5c39545873Bd663b0bB0716ED87dE0E399Aae419" "time"="2025-01-24 12:51:07.416074" "level"="info" "logger"="node" "msg"="using overlay address" "address"="74539eab1dbd5c722bb8ba10cef55f715e38f298b706fb1866af49f4fd15d8d3" "time"="2025-01-24 12:51:07.416149" "level"="info" "logger"="node" "msg"="starting with a disabled chain backend" "time"="2025-01-24 12:51:07.416242" "level"="info" "logger"="node" "msg"="using chain with network network" "chain_id"=100 "network_id"=1 "time"="2025-01-24 12:51:07.428047" "level"="info" "logger"="node" "msg"="starting debug & api server" "address"="127.0.0.1:1633" "time"="2025-01-24 12:51:07.464425" "level"="info" "logger"="node" "msg"="using datadir" "path"="/root/.bee" "time"="2025-01-24 12:51:07.486853" "level"="info" "logger"="migration-RefCountSizeInc" "msg"="starting migration of replacing chunkstore items to increase refCnt capacity" "time"="2025-01-24 12:51:07.486921" "level"="info" "logger"="migration-RefCountSizeInc" "msg"="migration complete" "time"="2025-01-24 12:51:07.489133" "level"="info" "logger"="node" "msg"="starting reserve repair tool, do not interrupt or kill the process..." "time"="2025-01-24 12:51:07.489346" "level"="info" "logger"="node" "msg"="removed all bin index entries" "time"="2025-01-24 12:51:07.489430" "level"="info" "logger"="node" "msg"="removed all chunk bin items" "total_entries"=0 "time"="2025-01-24 12:51:07.489482" "level"="info" "logger"="node" "msg"="counted all batch radius entries" "total_entries"=0 "time"="2025-01-24 12:51:07.489520" "level"="info" "logger"="node" "msg"="parallel workers" "count"=2 "time"="2025-01-24 12:51:07.489612" "level"="info" "logger"="node" "msg"="migrated all chunk entries" "new_size"=0 "missing_chunks"=0 "invalid_sharky_chunks"=0 "time"="2025-01-24 12:51:07.489659" "level"="info" "logger"="migration-step-04" "msg"="starting sharky recovery" "time"="2025-01-24 12:51:07.514853" "level"="info" "logger"="migration-step-04" "msg"="finished sharky recovery" "time"="2025-01-24 12:51:07.515253" "level"="info" "logger"="migration-step-05" "msg"="start removing upload items" "time"="2025-01-24 12:51:07.515374" "level"="info" "logger"="migration-step-05" "msg"="finished removing upload items" "time"="2025-01-24 12:51:07.515434" "level"="info" "logger"="migration-step-06" "msg"="start adding stampHash to BatchRadiusItems, ChunkBinItems and StampIndexItems" "time"="2025-01-24 12:51:07.515571" "level"="info" "logger"="migration-step-06" "msg"="finished migrating items" "seen"=0 "migrated"=0 "time"="2025-01-24 12:51:07.517270" "level"="info" "logger"="node" "msg"="starting in ultra-light mode" ``` ## Fund and Stake Running a full node for the purpose of earning xBZZ by sharing disk space and participating in the redistribution game requires a minimum of 10 xBZZ and a small amount of xDAI (for initializing the chequebook contract and for paying for redistribution-related transactions). While running a light node requires a small amount of xDAI to pay for initializing the chequebook contract and a smaller amount of xBZZ to pay for uploads and downloads. ### Fund node Check the logs from the previous step. Look for the line which says: ``` "time"="2024-09-24 18:15:34.520716" "level"="info" "logger"="node" "msg"="using ethereum address" "address"="0x1A801dd3ec955E905ca424a85C3423599bfb0E66" ``` That address is your node's address on Gnosis Chain which needs to be funded with xDAI (and also xBZZ if you plan on doing any uploading or on staking). Copy it and save it for the next step. You can also use the following command: ```bash curl -s localhost:1633/addresses | jq .ethereum ``` Which will return your node's address: ```bash "0x1A801dd3ec955E905ca424a85C3423599bfb0E66" ``` ***How Much to Send?*** Only a very small amount of xDAI is needed to get started, 0.1 xDAI is more than enough. For very small short term uploads you can start with ~0.2 xBZZ, but the required amount will scale up with the volume and duration of storage required. You will also need at least 10 xBZZ if you plan on staking. ### Initialize full node After sending the required tokens of ~0.1 xDAI and 10 xBZZ (or a smaller amount of xBZZ if you don't plan on staking) to your node's Gnosis Chain address, close the bee process in your terminal (`Ctrl + C`). Then start it again with the same command: ```bash bee start \ --password flummoxedgranitecarrot \ --full-node \ --swap-enable \ --api-addr 127.0.0.1:1633 \ --blockchain-rpc-endpoint https://xdai.fairdatasociety.org ``` After funding and restarting your node, the logs printed to the terminal should look something like this: ```bash Welcome to Swarm.... Bzzz Bzzzz Bzzzz \ / \ o ^ o / \ ( ) / ____________(%%%%%%%)____________ ( / / )%%%%%%%( \ \ ) (___/___/__/ \__\___\___) ( / /(%%%%%%%)\ \ ) (__/___/ (%%%%%%%) \___\__) /( )\ / (%%%%%) \ (%%%) ! DISCLAIMER: This software is provided to you "as is", use at your own risk and without warranties of any kind. It is your responsibility to read and understand how Swarm works and the implications of running this software. The usage of Bee involves various risks, including, but not limited to: damage to hardware or loss of funds associated with the Ethereum account connected to your node. No developers or entity involved will be liable for any claims and damages associated with your use, inability to use, or your interaction with other nodes or the software. version: 2.2.0-06a0aca7 - planned to be supported until 11 December 2024, please follow https://ethswarm.org/ "time"="2024-09-24 18:57:16.710417" "level"="info" "logger"="node" "msg"="bee version" "version"="2.2.0-06a0aca7" "time"="2024-09-24 18:57:16.760154" "level"="info" "logger"="node" "msg"="swarm public key" "public_key"="0373fe2ab33ab836635fc35864cf708fa0f4a775c0cf76ca851551e7787b58d040" "time"="2024-09-24 18:57:16.854594" "level"="info" "logger"="node" "msg"="pss public key" "public_key"="03a341032724f1f9bb04f1d9b22607db485cccd74174331c701f3a6957d94d95c1" "time"="2024-09-24 18:57:16.854651" "level"="info" "logger"="node" "msg"="using ethereum address" "address"="0x1A801dd3ec955E905ca424a85C3423599bfb0E66" "time"="2024-09-24 18:57:16.866697" "level"="info" "logger"="node" "msg"="using overlay address" "address"="22d502d022de0f8e9d477bc61144d0d842d9d82b8241568c6fe4e41f0b466615" "time"="2024-09-24 18:57:16.866730" "level"="info" "logger"="node" "msg"="starting with an enabled chain backend" "time"="2024-09-24 18:57:17.485408" "level"="info" "logger"="node" "msg"="connected to blockchain backend" "version"="erigon/2.60.1/linux-amd64/go1.21.5" "time"="2024-09-24 18:57:17.672282" "level"="info" "logger"="node" "msg"="using chain with network network" "chain_id"=100 "network_id"=1 "time"="2024-09-24 18:57:17.686479" "level"="info" "logger"="node" "msg"="starting debug & api server" "address"="127.0.0.1:1633" "time"="2024-09-24 18:57:18.065029" "level"="info" "logger"="node" "msg"="using default factory address" "chain_id"=100 "factory_address"="0xC2d5A532cf69AA9A1378737D8ccDEF884B6E7420" "time"="2024-09-24 18:57:18.252410" "level"="info" "logger"="node/chequebook" "msg"="no chequebook found, deploying new one." "time"="2024-09-24 18:57:19.576100" "level"="info" "logger"="node/chequebook" "msg"="deploying new chequebook" "tx"="0xf7bc9c5b04e96954c7f70cecfe717cad9cdc5d64b6ec080b2cbe712166ce262a" "time"="2024-09-24 18:57:27.619377" "level"="info" "logger"="node/transaction" "msg"="pending transaction confirmed" "sender_address"="0x1A801dd3ec955E905ca424a85C3423599bfb0E66" "tx"="0xf7bc9c5b04e96954c7f70cecfe717cad9cdc5d64b6ec080b2cbe712166ce262a" "time"="2024-09-24 18:57:27.619437" "level"="info" "logger"="node/chequebook" "msg"="chequebook deployed" "chequebook_address"="0x261a07a63dC1e7200d51106155C8929b432181fb" ``` Here we can see that after our node has been funded, it was able to issue the transactions for deploying the chequebook contract, which is a prerequisite for running a staking node. Next your node will begin to sync [postage stamp data](./../../develop/tools-and-features/buy-a-stamp-batch.md), which can take ~5 to 10 minutes. You will see this log message while your node is syncing postage stamp data: ```bash "time"="2024-09-24 22:21:19.664897" "level"="info" "logger"="node" "msg"="waiting to sync postage contract data, this may take a while... more info available in Debug loglevel" ``` After your node finishes syncing postage stamp data it will start in full node mode and begin to sync all the chunks of data it is responsible for storing as a full node: ```bash "time"="2024-09-24 22:30:19.154067" "level"="info" "logger"="node" "msg"="starting in full mode" "time"="2024-09-24 22:30:19.155320" "level"="info" "logger"="node/multiresolver" "msg"="name resolver: no name resolution service provided" "time"="2024-09-24 22:30:19.341032" "level"="info" "logger"="node/storageincentives" "msg"="entered new phase" "phase"="reveal" "round"=237974 "block"=36172090 "time"="2024-09-24 22:30:33.610825" "level"="info" "logger"="node/kademlia" "msg"="disconnected peer" "peer_address"="6ceb30c7afc11716f866d19b7eeda9836757031ed056b61961e949f6e705b49e" ``` This process can take a while, even up to several hours depending on your system and network. You can check the progress of your node through the logs which print out to the Bee API: You check your node's progress with the `/status` endpoint: :::info The [`jq` utility](https://jqlang.org/) jq utility formats API responses for easier reading: * Install it using your system’s package manager. * If you don't want to use it, remove `| jq` from all commands. ::: ```bash curl -s http://localhost:1633/status | jq ``` ```bash { "overlay": "22dc155fe072e131449ec7ea2f77de16f4735f06257ebaa5daf2fdcf14267fd9", "proximity": 256, "beeMode": "full", "reserveSize": 686217, "reserveSizeWithinRadius": 321888, "pullsyncRate": 497.8747754074074, "storageRadius": 11, "connectedPeers": 148, "neighborhoodSize": 4, "batchCommitment": 74510761984, "isReachable": false, "lastSyncedBlock": 36172390 } ``` We can see that our node has not yet finished syncing chunks since the `pullsyncRate` is around 497 chunks per second. Once the node is fully synced, this value will go to zero. However, we do not need to wait until our node is fully synced in order to stake our node, so we can now move immediately to the next step. ### Stake node Now we're ready to stake. We'll slightly modify our startup command so that it runs in the background instead of taking control of our terminal: ```bash nohup bee start \ --password flummoxedgranitecarrot \ --full-node \ --swap-enable \ --api-addr 127.0.0.1:1633 \ --blockchain-rpc-endpoint https://xdai.fairdatasociety.org > bee.log 2>&1 & ``` :::info 1. **`nohup`**: `nohup` prevents the `bee start` process from stopping when the terminal closes. 2. **`> bee.log 2>&1`**: Redirects both standard output and standard error to a log file called `bee.log`. 3. **`&`**: This sends the process to the background, allowing the terminal to be used for other commands while the Bee node continues running. ::: Let's check the Bee API to confirm the node is running: ``` curl localhost:1633 ``` If the node is running we should see: ``` Ethereum Swarm Bee ``` Now with our node properly running in the background, we're ready to stake our node. You can use the following command to stake 10 xBZZ: ```bash curl -XPOST localhost:1633/stake/100000000000000000 ``` If the staking transaction is successful a `txHash` will be returned: ``` {"txHash":"0x258d64720fe7abade794f14ef3261534ff823ef3e2e0011c431c31aea75c2dd5"} ``` We can also confirm that our node has been staked with the `/stake` endpoint: ```bash curl localhost:1633/stake ``` The results will be displayed in PLUR units (1 PLUR is equal to 1e-16 xBZZ). If you have properly staked the minimum 10 xBZZ, you should see the output below: ```bash {"stakedAmount":"100000000000000000"} ``` Congratulations! You have now installed your Bee node and are connected to the network as a full staking node. Your node will now be in the process of syncing chunks from the network. Once the node is fully synced, your node will finally be eligible to earn staking rewards. ### Set Target Neighborhood When installing your Bee node it will automatically be assigned a neighborhood. However, when running a full node with staking there are benefits to periodically updating your node's neighborhood. Learn more about why and how to set your node's target neighborhood [here](./set-target-neighborhood.md). ### Logs and monitoring :::info You can learn more about Bee logs [here](./../working-with-bee/logs-and-files.md). ::: With our previously modified command, our Bee node will now be running in the background and the logs will be written to the `bee.log` file. To review our node's logs we can simply view the file contents: ```bash cat bee.log ``` The file will continue to update with all the latest logs as they are output: ```bash "time"="2024-09-27 18:05:34.096641" "level"="info" "logger"="node/kademlia" "msg"="connected to peer" "peer_address"="03b48e678938d63c0761c74a805fbe0446684c9c417330c2bec600ecfd6c492f" "proximity_order"=8 "time"="2024-09-27 18:05:35.168425" "level"="info" "logger"="node/kademlia" "msg"="connected to peer" "peer_address"="0e9388fff473a9c74535337c32cc74d8f921514d2635d0c4a49c6e8022f5594e" "proximity_order"=4 "time"="2024-09-27 18:05:35.532723" "level"="info" "logger"="node/kademlia" "msg"="disconnected peer" "peer_address"="3c195cd8882ee537d170e92d959ad6bd72a76a50097a671c72646e83b45a1832" ``` There are many different ways to monitor your Bee node's process, but one convenient way to do so is the [bashtop command line tool](https://github.com/aristocratos/bashtop). The method of [installation](https://github.com/aristocratos/bashtop?tab=readme-ov-file#installation) will vary depending on your system. After installation, we can launch it with the `bashtop` command: ```bash bashtop ``` ![](/img/bashtop_01.png) We can use the `f` key to filter for our Bee node's specific process by searching for the `bee` keyword (use the arrow keys to navigate and `enter` to select). From here we can view info about our node's process, or shut it down using the `t` key (for "terminate"). ![](/img/bashtop_02.png) **Checking the Node's status with the Bee API** To check your node's status as a staking node, we can use the `/redistributionstate` endpoint: ```bash curl -s http://localhost:1633/redistributionstate | jq ``` Below is the output for a node that has been running for several days: ```bash { "minimumGasFunds": "11080889201250000", "hasSufficientFunds": true, "isFrozen": false, "isFullySynced": true, "phase": "claim", "round": 212859, "lastWonRound": 207391, "lastPlayedRound": 210941, "lastFrozenRound": 210942, "lastSelectedRound": 212553, "lastSampleDuration": 491687776653, "block": 32354719, "reward": "1804537795127017472", "fees": "592679945236926714", "isHealthy": true } ``` For a complete breakdown of this output, check out [this section in the Bee docs](./../working-with-bee/bee-api.md#redistributionstate). You can read more other important endpoints for monitoring your Bee node in the [official Bee docs](./../working-with-bee/bee-api.md), and you can find complete information about all available endpoints in [the API reference docs](/api/). ## Back Up Keys Once your node is up and running, make sure to [back up your keys](./../working-with-bee/backups.md). ## Getting help The CLI has built-in documentation. Running `bee` gives you an entry point to the documentation. Running `bee start -h` or `bee start --help` will tell you how you can configure your Bee node via the command line arguments. You may also check out the [configuration guide](./../working-with-bee/configuration.md), or simply run your Bee terminal command with the `--help` flag, eg. `bee start --help` or `bee --help`. ## Next Steps to Consider ### Access the Swarm If you'd like to start uploading or downloading files to Swarm, [start here](./../../develop/introduction.md). ### Explore the API The [Bee API](./../working-with-bee/bee-api.md) is the primary method for interacting with Bee and getting information about Bee. After installing Bee and getting it up and running, it's a good idea to start getting familiar with the API. ### Run a hive! If you would like to run a hive of many Bees, check out the [hive operators](./hive.md) section for information on how to operate and monitor many Bees at once. ### Start building DAPPs on Swarm If you would like to start building decentralised applications on Swarm, check out our section for [developing with Bee](./../../develop/introduction.md). --- ## Backups Backing up your Bee node involves copying and saving files from the data directory specified in the `dat-dir` configuration option, along with the node's password. The details of where and how this option is specified will vary depending on the type of [configuration method](./configuration.md) used (YAML file, command line flag, or environment variable). :::caution A node's password may be specified in several different locations. It can be specified either through the `password` option or the `password-file` option. For a backup, you will need to either copy the `password` option value, or copy the file itself from the location specified by the `password-file` option. Don't forget - it's not a backup until you're sure the backup files work! Make sure to test restoring from backup files and password to prevent loss of assets due to data loss or corruption. ::: ## Bee Files A full Bee node backup includes the `keys`, `localstore`, `stamperstore`, `statestore`, and `password` files. The node should be stopped before taking a backup and not restarted until restoring the node from the backup to prevent the node from getting out of sync with the network. Key data from the `keys` directory allows access to Bee node's Gnosis account (provided that you have also made sure to back the password for your keys). If your keys and password are lost or stolen it could lead to the loss of all assets in that account. The `stamperstore` contains postage stamp data. If lost, previously purchased postage stamps will become unusable. ### Statestore and Localstore. The `statestore` retains data related to its operation, and the `localstore` contains chunks locally which are frequently requested, pinned in the node, or are in the node's neighborhood of responsibility. :::info As the data in `statestore` and `localstore` continually changes during normal operation of a node, when taking a backup the node should first be stopped and not re-connected to the Swarm network until restoring from the backup (otherwise the `statestore` and `localstore` files will get out of sync with the network). It is possible to restore using out of sync `statestore` and `localstore` files, however it may lead to data loss or unexpected behavior related to chunk uploads, postage stamps, and more. ::: ### Stamperstore The `stamperstore` contains postage stamp batch related data, and so is important to include in your backup if you have purchased any postage batches which you wish to continue using. ### Keys The `keys` directory contains the following key files: * `libp2p.key` * `libp2p_v2.key` * `pss.key` * `swarm.key` These keys are generated during the Bee node's initialisation and are required for maintaining access to your node. :::danger The `swarm.key` file grants full control over your node's Gnosis Chain account. If lost, you cannot recover funds. If stolen, your assets can be drained. ::: :::info To use `swarm.key` to manage the Gnosis account for a node through Metamask or other wallets, [exportSwarmKeys](https://github.com/ethersphere/exportSwarmKey) can be used to convert `swarm.key` to a compatible format. ::: ### Data Directory Structure The data directory contains four directories. Its default location depends on the node install method and startup method used. ``` ├── kademlia-metrics │   └── ... ├── keys │   ├── libp2p.key │   ├── libp2p_v2.key │   ├── pss.key │   └── swarm.key ├── localstore │   ├── indexstore │   └── sharky ├── password ├── stamperstore │   └── ... └── statestore │   └── ... ``` ## Data Directory Locations The default data directory for your Bee node will depend on the installation method used. :::caution If Bee is installed to run as a service using a package manager such as `apt` or `yum`, then it can be started using your system's services manager such as `systemctl` using a command like `systemctl start bee`. However, after installing with a package manager, Bee can also by started using the `bee start` command used for running Bee with a shell script / binary install. When the `bee start` command is run, it will create a SECOND data directory alongside the default data directory for your package manager at the same directory it would for the shell script installation: ``` /home//.bee ``` In that case, you would have two separate data directories in two different locations, and the directory used will depend on whether you start your node using a service manager like `systemctl` or the `bee start` command. If you installed Bee via a package manager but sometimes start it manually, you may have two separate data directories: * System service (`systemctl start bee`) → Uses `/var/lib/bee`. * Manual start (`bee start`) → Uses `/home//.bee`. *The exact directory will differ depending on your system. See [Configuration page](./configuration.md#default-data-and-config-directories).* ::: ### *apt* and *yum / rpm* Package Managers Default `data-dir` location: ``` /var/lib/bee ``` ### Homebrew (amd64) Default `data-dir` location: ``` /usr/local/var/lib/swarm-bee ``` ### Homebrew (arm64) Default `data-dir` location: ``` /opt/homebrew/var/lib/swarm-bee ``` ### *scoop* Package Manager Default `data-dir` location: ``` ./data ``` ### Shell Script & Binary Install If you installed Bee using the [automated shell script](./../installation/shell-script.md) or by [building Bee from source](./../installation/build-from-source.md), your data directory will typically be located at: ```bash /home//.bee ``` ### Docker Default `data-dir` location: ``` /home/bee/.bee ``` ## Back-up your node data Copy entire `bee` data folder to create a full backup. This will do a full backup of `kademlia-metrics`, `keys`, `statestore`, `stamperstore`, `password`, and `localstore`, files into a newly created `/backup` directory. Make sure to save the backup directory to a safe location. :::tip For a more lightweight backup, you can remove `localstore` and `localstore`. You can safely restore your node from the remaining files. ::: ``` mkdir backup sudo cp -r /var/lib/bee/ backup ``` ## Back-up your password Depending on your [configuration](./configuration.md) method, your password may be located in a variety of different locations. If you use a `.yaml` file for your configuration, then it might be found directly under the `password` option, or it could be that the location of your password file is recorded by the `password-file` option. In either case, make sure to record the password somewhere safe or include the password file as a part of your backup. The same applies to other configuration methods. If you use environment variables for specifying your configuration options, your password itself will likely be specified in a `.env` file somewhere which contains either the password itself in the `BEE_PASSWORD` variable or the location of your password file in the `BEE_PASSWORD_FILE` variable. The same again holds true for the command line flag method. Make sure you have the password you use with the `--password` command line flag or the password file specified by the `--password-file` flag saved in your backup. ## Back-up blockchain keys only If you only need to export your node's blockchain keys, you need to export the `swarm.key` UTC / JSON keystore file and the `password` file used to encrypt it. First create a directory for your keys and then copy your keys to that directory. ```bash mkdir keystore sudo cp -r /var/lib/bee/keys/swarm.key /var/lib/bee/password keystore ``` ## Metamask Import If you wish to import your Bee node’s Gnosis Chain account into Metamask, find your `swarm.key` and `password`, then follow these steps: ## View key and password for wallet import ```bash sudo cat /var/lib/bee/keys/swarm.key sudo cat /var/lib/bee/password ``` :::info Note that `swarm.key` is in UTC / JSON keystores format and is encrypted by default by your password file inside the `/bee` directory. Make sure to export both the `swarm.key` file and the `password` file in order to secure your wallet. If you need your private key exported from the keystore file, you may use one of a variety of Ethereum wallets which support exporting private keys from UTC files (such as [Metamask](https://metamask.io/), however we offer no guarantees for any software, make sure you trust it completely before using it). ::: ## Get private key from keystore and password To import to Metamask: 1. View and copy the contents of your exported `swarm.key` and `password` files 2. Go to Metamask and click "Account 1" --> "Import Account" 3. Choose the "Select Type" dropdown menu and choose "JSON file" 4. Paste the password (Make sure to do this first) 5. Upload exported JSON file 6. Click "Import" To export your private key: 1. Go to Metamask and click "Account 1" to view the dropdown menu of all accounts 2. Click the three dots next to the account you want to export 3. Click "Account details" 4. Click "Show private key" 5. Enter your Metamask password (not your keystore password) 6. Copy your private key to a safe location ## Restore from backup :::danger Before restoring, make sure to check for any old node data from a previous node which has not yet been backed up, and back it up if needed. ::: :::tip The specific directories and commands for restoring will depend on which install method and system is used. The instructions below are for a Linux package manager based installation. See the [configuration section](./configuration.md#default-data-and-config-directories) more more details about default file locations. ::: 1. After [uninstalling](./uninstalling-bee.md) any existing Bee installations, perform a new [installation](./../installation/getting-started.md#installation-methods). 1. Remove any existing Bee node data before restoring. This prevents conflicts with old files: ``` sudo rm -r /var/lib/bee ``` 1. Navigate to backup directory and copy files to data folder. ``` sudo cp -r //. /var/lib/bee ``` 1. Revert ownership of the data folder. ``` sudo chown -R bee:bee /var/lib/bee ``` 1. Restart `bee` and check logs. ``` sudo systemctl restart bee sudo journalctl --lines=100 --follow --unit bee ``` --- ## Bcrypt hashing utility In order to generate a valid admin password hash you can use any available bcrypt compatible tools, both [online](https://bcrypt-generator.com/) and offline (htpasswd). For convenience Bee also provides a method to generate and validate password hashes: ```sh $ bee bcrypt super$ecret $2a$10$eZP5YuhJq2k8DFmj9UJGWOIjDtXu6NcAQMrz7Zj1bgIVBcHA3bU5u $ bee bcrypt --check super$ecret '$2a$10$eZP5YuhJq2k8DFmj9UJGWOIjDtXu6NcAQMrz7Zj1bgIVBcHA3bU5u' OK: password hash matches provided plain text ``` :::info When validating a hash don't forget about quotes - the ($) hash prefix might interfere with your terminal. ::: --- ## Bee API The Bee HTTP API is the primary interface to a running Bee node. API-endpoints can be queried using familiar HTTP requests, and will respond with semantically accurate [HTTP status and error codes](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status) as well as data payloads in [JSON](https://www.json.org/json-en.html) format where appropriate. The Bee API provides full access to all core functionalities of a Bee node, including uploading, downloading, staking, postage stamp batch purchasing and management, and node monitoring. By default, it runs on port `:1633`. :::danger Make sure that your api-addr (default 1633) is never exposed to the internet. If you do not have a firewall or other security measures in place, manually setting your Bee API address from the default `1633` to `127.0.0.1:1633` is strongly recommended to prevent unauthorized access. You may also consider using the [Gateway Proxy tool](./../../develop/tools-and-features/gateway-proxy.md) to protect your node's API endpoint. ::: Detailed information about Bee API endpoints can be found in the [API reference docs](/api/). ## Interacting With the API You can interact with the Bee API using standard HTTP requests, allowing you to programmatically access all of your Bee node's various functions such as [purchasing stamp batches](./../../develop/tools-and-features/buy-a-stamp-batch.md), [uploading and downloading](./../../develop/upload-and-download.md), [staking](./staking.md), and more. ### Alternatives for Working with the API For developers, the [Bee JS library](./../../develop/tools-and-features/bee-js.md) offers a more convenient way to interact with the API in a NodeJS environment. For many other common use cases, you may prefer to make use of the [Swarm CLI](./swarm-cli.md) tool, as it offers a convenient command line based interface for interacting with your node's API. ## Exploring Node Status After installing and starting up your node, we can begin to understand the node's status by interacting with the API. For example, to determine how many nodes your Bee node is currently connected to, run: ```bash curl -s http://localhost:1633/peers | jq '.peers | length' ``` ``` 23 ``` Great! We can see that we are currently connected with 23 other nodes! :::info Here we are using the `jq` command line utility to count the amount of objects in the `peers` array in the JSON response we have received from our API, learn more about how to install and use `jq` [here](https://jqlang.org/). ::: Let's review a handful of endpoints which will provide you with important information relevant to detecting and diagnosing problems with your nodes. ### _/status_ The `/status` endpoint returns a quick summary of some important metrics for your node. ```bash curl -s http://localhost:1633/status | jq ``` ```json { "overlay": "1e2054bec3e681aeb0b365a1f9a574a03782176bd3ec0bcf810ebcaf551e4070", "proximity": 256, "beeMode": "full", "reserveSize": 3215597, "reserveSizeWithinRadius": 3215806, "pullsyncRate": 1.5622222222222222, "storageRadius": 10, "connectedPeers": 89, "neighborhoodSize": 12, "batchCommitment": 11615207424, "isReachable": true, "lastSyncedBlock": 41786200, "committedDepth": 10, "isWarmingUp": false } ``` - `"overlay"` - Your node's overlay address. - `"proximity"` - The proximity order (PO), representing how closely related this node is to your own node in Swarm's Kademlia network. - `"beeMode"` - The mode of your node, can be `"full"`, `"light"`, or `"ultraLight"`. - `"reserveSize"` - The number of chunks your node is currently storing in its reserve. This value should be roughly similar across nodes in the network. It should be identical for nodes within the same neighborhood. - `"reserveSizeWithinRadius"` - The number of chunks your node is currently storing which fall within its storage radius. - `"pullsyncRate"` - The rate at which your node is currently syncing chunks from other nodes in the network. - `"storageRadius"` - The storage radius (radius of responsibility ) is the proximity order of chunks for which your node is responsible for storing. It should generally match the radius shown on [Swarmscan](https://swarmscan.io/neighborhoods). - `"connectedPeers"` - The number of peers your node is connected to. - `"neighborhoodSize"` - The number of total neighbors in your neighborhood, not including your own node. The more nodes in your neighborhood, the lower your chance of winning rewards as a staking node. - `"batchCommitment"` - The total number of chunks which would be stored on the Swarm network if 100% of all postage batches were fully utilised. - `"isReachable"` - Whether or not your node is reachable on the p2p API by other nodes on the Swarm network (port 1634 by default). - `"lastSyncedBlock"` - The last block number from the connected blockchain that your node has synced up to. - `"committedDepth"` - The storage depth currently committed by your node, which defines how much of your reserve is actually being used to store chunks. Is equal to `"storageRadius"` plus the [doubling factor](./staking.md#reserve-doubling) specified in the `reserve-capacity-doubling` option (which is zero by default). - `"isWarmingUp"` - Indicates whether your node is still in the warm-up phase (building up its reserve and syncing with the network) or has reached normal operation. ### _/status/peers_ The `/status/peers` endpoint returns information about all the peers of the node making the request. The type of the object returned is the same as that returned from the `/status` endpoint. This endpoint is useful for diagnosing syncing / availability issues with your node. The list is sorted by Kademlia proximity, not geographical distance. Nodes with lower PO values are further away, while higher PO values indicate closer neighbors. The most distant nodes with PO (proximity order) of zero are at the top of the list and the closest nodes with higher POs at the bottom of the list. The nodes at the bottom of the list with a PO equal or greater than the storage depth make up the nodes in your own node's neighborhood. It's possible that not all nodes in your neighborhood will appear in this list each time you call the endpoint if the connection between your nodes and the rest of the nodes in the neighborhood is not stable. Here are the last few entries: ```bash curl -s http://localhost:1633/status/peers | jq ``` ```json ... { "overlay": "1e1547d0d629469ff0d8fd2cbb6435df8fd913f2e948f177d733356d784b7ea4", "proximity": 10, "beeMode": "full", "reserveSize": 3217677, "reserveSizeWithinRadius": 3215613, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 153, "neighborhoodSize": 12, "batchCommitment": 11615207424, "isReachable": true, "lastSyncedBlock": 41786375, "committedDepth": 10, "isWarmingUp": false }, { "overlay": "1e09531ee3d8031b130b1c7d530dac26f57d2b9cfd368a979ef227331deb2ae5", "proximity": 10, "beeMode": "full", "reserveSize": 3215934, "reserveSizeWithinRadius": 3215613, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 166, "neighborhoodSize": 12, "batchCommitment": 11615207424, "isReachable": true, "lastSyncedBlock": 41786375, "committedDepth": 10, "isWarmingUp": false }, { "overlay": "1e1ad7975d88430b8ede359ca231e73aaffaeefe35d6f32e709ff37dc3028eaa", "proximity": 10, "beeMode": "full", "reserveSize": 3215364, "reserveSizeWithinRadius": 3215344, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 150, "neighborhoodSize": 12, "batchCommitment": 11614158848, "isReachable": true, "lastSyncedBlock": 41786375, "committedDepth": 10, "isWarmingUp": false }, { "overlay": "1e30c8fe93339f8637a339b5d4d85ec42731a193be8987c6457f4ea72c93cfb7", "proximity": 11, "beeMode": "full", "reserveSize": 3218783, "reserveSizeWithinRadius": 3215613, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 165, "neighborhoodSize": 12, "batchCommitment": 11615207424, "isReachable": true, "lastSyncedBlock": 41786375, "committedDepth": 10, "isWarmingUp": false }, { "overlay": "1e3c168d12e0f590640454c01e2825522ca60eb0a1c7dfaac9da2329e9d87300", "proximity": 11, "beeMode": "full", "reserveSize": 3215635, "reserveSizeWithinRadius": 3215613, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 169, "neighborhoodSize": 12, "batchCommitment": 11615207424, "isReachable": true, "lastSyncedBlock": 41786375, "committedDepth": 10, "isWarmingUp": false }, { "overlay": "1e3f2e9b0f6d45aa1fd710e7fca4a7890d2cbde829cd2722674ab120544e3772", "proximity": 11, "beeMode": "full", "reserveSize": 3215645, "reserveSizeWithinRadius": 3215613, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 163, "neighborhoodSize": 12, "batchCommitment": 11615207424, "isReachable": true, "lastSyncedBlock": 41786375, "committedDepth": 10, "isWarmingUp": false }, { "overlay": "1e288371f2e3c3325c1a3af5008d7c81fa4ab1d176e1c6bbb3f9ace4655dc05d", "proximity": 12, "beeMode": "full", "reserveSize": 3215644, "reserveSizeWithinRadius": 3215613, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 165, "neighborhoodSize": 12, "batchCommitment": 11615207424, "isReachable": true, "lastSyncedBlock": 41786375, "committedDepth": 10, "isWarmingUp": false }, { "overlay": "1e2c2b11a118a0be240af19421a8a323610869247625fa28a7590d765a21c566", "proximity": 12, "beeMode": "full", "reserveSize": 3215633, "reserveSizeWithinRadius": 3215613, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 172, "neighborhoodSize": 12, "batchCommitment": 11615207424, "isReachable": true, "lastSyncedBlock": 41786370, "committedDepth": 10, "isWarmingUp": false }, { "overlay": "1e20ef01ddab9112a9a26618d901c761f20d8bcb8328c143ab13e9846be9ad82", "proximity": 16, "beeMode": "full", "reserveSize": 3215652, "reserveSizeWithinRadius": 3215613, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 164, "neighborhoodSize": 12, "batchCommitment": 11615207424, "isReachable": true, "lastSyncedBlock": 41786375, "committedDepth": 10, "isWarmingUp": false } ] } ``` And we can compare these entries to our own node's `/status` results for diagnostic purposes: ```bash curl -s http://localhost:1633/status | jq ``` ```json { "overlay": "1e2054bec3e681aeb0b365a1f9a574a03782176bd3ec0bcf810ebcaf551e4070", "proximity": 256, "beeMode": "full", "reserveSize": 3215597, "reserveSizeWithinRadius": 3215806, "pullsyncRate": 1.5622222222222222, "storageRadius": 10, "connectedPeers": 89, "neighborhoodSize": 12, "batchCommitment": 11615207424, "isReachable": true, "lastSyncedBlock": 41786200, "committedDepth": 10, "isWarmingUp": false } ``` From the results we can see that our node's neighborhood size and batch commitment are generally in line with other nodes in the same neighborhood. Any significant discrepancy may indicate a problem with your node. ### _/redistributionstate_ This endpoint provides an overview of values related to storage fee redistribution game (in other words, staking rewards). You can use this endpoint to check whether or not your node is participating properly in the redistribution game. ```bash curl -s http://localhost:1633/redistributionstate | jq ``` ```json { "minimumGasFunds": "11080889201250000", "hasSufficientFunds": true, "isFrozen": false, "isFullySynced": true, "phase": "claim", "round": 212859, "lastWonRound": 207391, "lastPlayedRound": 210941, "lastFrozenRound": 210942, "lastSelectedRound": 212553, "lastSampleDuration": 491687776653, "block": 32354719, "reward": "1804537795127017472", "fees": "592679945236926714", "isHealthy": true } ``` * `"minimumGasFunds"` - The minimum required xDAI denominated in wei (1 xDAI = 10^18 wei) required for a node to participate in the redistribution game. * `"hasSufficientFunds"` - Whether your node has at least the `"minimumGasFunds"` amount of xDAI. * `"isFrozen"` - Indicates if your node is frozen, which may occur for [several reasons](./staking.md#diagnosing-freezing-issues). * `"isFullySynced"` - Whether your node has fully synced all the chunks in its `"storageRadius"` (the value returned from the `/reservestate` endpoint.) * `"phase"` - The current phase of the redistribution game (this does not indicate whether or not your node is participating in the current phase). * `"round"` - The current number of the round of the redistribution game. * `"lastWonRound"` - The last round number in which your node won the redistribution game. * `"lastPlayedRound"` - The last round number in which your node participating in the redistribution game. If this number matches the number of the current round shown in `"round"`, then your node is participating in the current round. * `"lastFrozenRound"` - The last round in which your node was frozen. * `"lastSelectedRound"` - The last round in which your node's neighborhood was selected. Note that it is possible for your node's neighborhood to be selected without your node playing in the redistribution game. This may potentially indicate your node's hardware is not sufficient to calculate the commitment hash fast enough. See [section on the `/rchash` endpoint](#rchash) for more information. * `"lastSampleDuration"` - The time it took for your node to calculate the sample commitment hash in nanoseconds. * `"block"` - current Gnosis block number * `"reward"` - The total all-time reward in PLUR earned by your node. * `"fees"` - The total amount in fees paid by your node denominated in xDAI wei. * `"isHealthy"` - a check of whether your node’s storage radius is the same as the most common radius from among its peer nodes ### _/reservestate_ This endpoint shows key information about the reserve state of your node. You can use it to identify problems with your node related to its reserve (whether it is syncing chunks properly into its reserve for example). ```bash curl -s http://localhost:1633/reservestate | jq ``` ```json { "radius": 15, "storageRadius": 10, "commitment": 134121783296, "reserveCapacityDoubling": 0 } ``` Let's take a look at each of these values: * `"radius"` - Represents the maximum storage radius assuming all postage stamp batches are fully utilized. * `"storageRadius"` - The radius of responsibility - the proximity order of chunks for which your node is responsible for storing. It should generally match the radius shown on [Swarmscan](https://swarmscan.io/neighborhoods). * `"commitment"` - The total number of chunks which would be stored on the Swarm network if 100% of all postage batches were fully utilised. * `"reserveCapacityDoubling"` - Indicates whether your node is currently using the reserve doubling mechanism. See [Reserve Doubling](./staking.md#reserve-doubling) for details. ### _/chainstate_ This endpoint relates to your node's interactions with the Swarm Smart contracts on the Gnosis Chain. ```bash curl -s http://localhost:1633/chainstate | jq { "chainTip": 41786513, "block": 41786505, "totalAmount": "293796491451", "currentPrice": "56774" } ``` * `"chainTip"` - The latest Gnosis Chain block number. Should be as high as or almost as high as the block number shown at [GnosisScan](https://gnosisscan.io/). * `"block"` - The latest block your node has fully synced from Gnosis Chain. If significantly behind `"chainTip"`, your node may still be catching up. Should be very close to `"chainTip"` if your node has already been operating for a while. * `"totalAmount"` - Cumulative value of all prices per chunk in PLUR for each block. * `"currentPrice"` - The price in PLUR to store a single chunk for each Gnosis Chain block. ### _/topology_ This endpoint allows you to explore the topology of your node within the Kademlia network. The results are split into 32 bins from bin_0 to bin_32. Each bin represents the nodes in the same neighborhood as your node at each proximity order from PO 0 to PO 32. As the output of this file can be very large, we save it to the `topology.json` file for easier inspection: ```bash curl -s http://localhost:1633/topology | jq '.' > topology.json ``` We open the file in vim for inspection: ```bash vim topology.json ``` The `/topology` endpoint provides insights into how your node is positioned within the Swarm network. The response starts with global network statistics, followed by detailed bin-by-bin peer connections (for 32 bins). Lets first look at the global stats: ```json "baseAddr": "da7e5cc3ed9a46b6e7491d3bf738535d98112641380cbed2e9ddfe4cf4fc01c4", "population": 20514, "connected": 176, "timestamp": "2024-02-08T20:57:03.815537925Z", "nnLowWatermark": 3, "depth": 10, "reachability": "Public", "networkAvailability": "Available", ... ``` - `"baseAddr"` - Your node's overlay address. - `"population"` - The total number of nodes your node has collected information about. This number should be around ####. If it is far higher or lower it likely indicates a problem. - `"connected"` - The total number of nodes your node is currently connected to. - `"timestamp"` - The time at which this topology snapshot was taken. - `"nnLowWatermark"` - ??? - `"depth"` - - `"reachability"` - `"networkAvailability"` After the first section are 32 sections, one for each bin. At the front of each of these sections is a summary of information about the respective bin followed two list, one of disconnected peers and the other of connected peers. Let's take a look at bin_10 as an example: ```json ... "bin_10": { "population": 3, // The total number of peers in this bin including both connected and disconnected peers. "connected": 2, // Number of connected peers "disconnectedPeers": [ //List of all disconnected peers { "address": "3e06e4667260c761f1b6a8539a99621c1af1f945e97667376c13b5f84984bcbc", "metrics": { "lastSeenTimestamp": 1707426772, "sessionConnectionRetry": 2, "connectionTotalDuration": 104619, "sessionConnectionDuration": 72, "sessionConnectionDirection": "outbound", "latencyEWMA": 849, "reachability": "Public", "healthy": true } } ], "connectedPeers": [ // List of all connected peers { "address": "3e09deca28d24a4c6dab9350dd0fb27a2333f03120b9f92f0ac0fd245707c9e3", "metrics": { "lastSeenTimestamp": 1707426766, "sessionConnectionRetry": 2, "connectionTotalDuration": 105059, "sessionConnectionDuration": 33, "sessionConnectionDirection": "outbound", "latencyEWMA": 899, "reachability": "Public", "healthy": true } }, { "address": "3e1cdf7b1072fcde264c75f70635b9c1e9c1623eab2de55a0380f17b07751955", "metrics": { "lastSeenTimestamp": 1707426741, "sessionConnectionRetry": 1, "connectionTotalDuration": 109216, "sessionConnectionDuration": 59, "sessionConnectionDirection": "outbound", "latencyEWMA": 948, "reachability": "Public", "healthy": true } } ] }, ``` ### _/node_ This endpoint returns info about options related to your node type and also displays your current node type. ```bash curl -s http://localhost:1633/node | jq ``` ```json { "beeMode": "full", "chequebookEnabled": true, "swapEnabled": true } ``` * `"beeMode"` - The mode of your node, can be `"full"`, `"light"`, or `"ultraLight"`. * `"chequebookEnabled"` - Whether or not your node's `chequebook-enable` option is set to `true`. * `"swapEnabled"` - Whether or not your node's `swap-enable` option is set to `true`. If your node is not operating in the correct mode, this can help you to diagnose whether you have set your options correctly. ### _/rchash_ Calling the `/rchash` endpoint triggers the generation of a reserve commitment hash, which is used in the [redistribution game](/docs/concepts/incentives/redistribution-game), and will report the amount of time it took to generate the hash. This is useful for getting a performance benchmark to ensure that your node's processor and disk are fast enough. The [`swarm-cli`](./swarm-cli.md) command doesn't require arguments. It reads the node's overlay address and committed depth, and derives the anchor and depth parameters from them. ```bash swarm-cli utility rchash ``` Pass `--depth` to benchmark against a depth other than the node's current one. The command gives as a result the time it took to generate the reserve commitment hash. ```bash Reserve sampling duration: 360.37808911 seconds ``` The `/rchash` endpoint has 3 parameters: `depth`, `anchor1`, and `anchor2`. For both anchor parameters, use the first 4 hex digits from your node's overlay address (which you can find from the `/addresses` endpoint). For depth, use your node's `committedDepth` from the `/status` endpoint. For nodes which do not use [reserve doubling](./staking.md#reserve-doubling), `committedDepth` is equal to `storageRadius`: ```text /rchash/{depth}/{anchor1}/{anchor2} ``` :::info anchor parameter details - The anchor parameters must match the prefix bits of the node's overlay address up to at least the current storage depth (with each hex digit equal to 4 bits). - The anchor parameters also must have an even number of digits. Therefore you can use the first four digits of your node's overlay address since it will work for depths up to depth 16, which will not be approached unless the depth increases up to depth 17, which is not likely to happen in the near future. If it does increase to depth 17, then the first 6 overlay digits should be used. ::: ```bash curl -sX GET http://localhost:1633/rchash/10/1e20/1e20 | jq ``` The response includes a `hash`, `proofs` (used by the redistribution smart contract), and `durationSeconds` which is the benchmark metric. Here is an example of a successful result: ```json { "hash": "a1d6e1700dff0c5259029c8a58904251855911eb298b45fab4b0c26d4de0fa5f", "proofs": { "proof1": { ... }, "proof2": { ... }, "proofLast": { ... } }, "durationSeconds": 287.52 } ``` The `durationSeconds` value should not exceed roughly 6 minutes (360 seconds). :::caution A single measurement is not a guarantee Sampling time scales with how full the node's reserve is within its radius, since the sampler walks every chunk in radius. Reserve occupancy depends on network conditions rather than on anything the operator sets, so a result measured against a half-full reserve says little about the same node once the reserve fills up. A node holding around 2M chunks can complete a sample in roughly half the time of one at the full default reserve capacity of about 4M chunks. Aim for a comfortable margin below 360 seconds; a marginal pass is not enough. ::: :::warning Slow results If `durationSeconds` is much longer than 360 seconds (for example, 1191 seconds / ~20 minutes), the node will likely fail to submit proofs in time during the redistribution game, resulting in missed rewards or freezing. The sampler reads every chunk in radius from local storage, so disk and processor speed are typically the bottlenecks. Upgrade to a faster SSD first, then consider a faster processor or more cores. ::: If while running the `/rchash` command there is an evictions related error such as the one below, try running the call to the `/rchash` endpoint again. ```text error: "level"="error" "logger"="node/storageincentives" "msg"="make sample" "error"="sampler: failed creating sample: sampler stopped due to ongoing evictions" ``` While evictions are a normal part of Bee's standard operation, the event of an eviction will interrupt the sampler process. ### _/health_ The `/health` endpoint provides a quick status check for your Bee node which simply indicates whether the node is operating or not. It is often used in tools like Docker and Kubernetes. ```bash curl -s http://localhost:1633/health | jq ``` ```json { "status": "ok", "version": "2.8.1-7cf53193", "apiVersion": "8.1.0" } ``` - `"status"` - "ok" if the server is responsive. - `"version"` - The version of your Bee node. You can find latest version by checking the [Bee github repo](https://github.com/ethersphere/bee). - `"apiVersion"` --- ## Cashing Out There are two different types of cashing out. The first type is cashing out xBZZ rewards earned from staking and providing storage services (this method also allows for withdrawal of the native xDAI token). The second type is for the withdrawal of xBZZ earned through bandwidth incentives (SWAP). Both types are explained below: ## Withdrawing xBZZ Rewards and Native xDAI You can withdraw xBZZ rewards or native xDAI tokens using the `/wallet/withdraw/` endpoint. The endpoint allows you to withdraw tokens to any address which you have whitelisted using the `withdrawal-addresses-whitelist` option. You can specify either a single address: ```yaml # withdrawal target addresses withdrawal-addresses-whitelist: 0x62d04588e282849d391ebff1b9884cb921b9b94a ``` Or an array of addresses: ```yaml # withdrawal target addresses withdrawal-addresses-whitelist: [ 0x62d04588e282849d391ebff1b9884cb921b9b94a, 0x71a5aae026e2ab87612a5824d492a095e7d790bf ] ``` The token you desire to withdraw is specified in the path directly: ```bash http://localhost:1633/wallet/withdraw/{coin} ``` For `coin`, you can use the value `NativeToken` for xDAI or `BZZ` for xBZZ. The `amount` query parameter is used to specify how much of the token you wish to withdraw. The value should be specified in the lowest denomination for each token (wei for xDAI and PLUR for xBZZ). The `address` query parameter is used to specify the target address to withdraw to. This address must be specified using the `withdrawal-addresses-whitelist` option in your configuration. The following command will withdraw a single PLUR of xBZZ to address 0x62d04588e282849d391ebff1b9884cb921b9b94a: ```bash curl -X POST "http://localhost:1633/wallet/withdraw/bzz?amount=1&address=0x62d04588e282849d391ebff1b9884cb921b9b94a" ``` ## Cashing out Cheques (SWAP) As your Bee forwards and serves chunks to its peers, it is rewarded in xBZZ in the form of cheques. Once these cheques accumulate sufficient value, you may _cash them out_ using Bee's API. This process transfers money from your peer's chequebooks into your own, which you can then withdraw to your wallet to do with as you please! :::important Do **not** cash out your cheques too regularly! Once a week is more than sufficient! Besides the transaction costs, this prevents and relieves unnecessary congestion on the blockchain. 💩 ::: :::info Learn more about how SWAP and other accounting protocols work by reading [The Book of Swarm](https://www.ethswarm.org/the-book-of-swarm-2.pdf). ::: Bee contains a rich set of features to enable you to query the current accounting state of your node. First, let's query our node's current balance by sending a POST request to the balances endpoint. ```bash curl localhost:1633/chequebook/balance | jq ``` ```json { "totalBalance": "10000000", "availableBalance": "9640360" } ``` It is also possible to examine your per-peer balances. ```bash curl localhost:1633/balances | jq ``` ```json { "balances": [ //... { "peer": "d0bf001e05014fa036af97f3d226bee253d2b147f540b6c2210947e5b7b409af", "balance": "-85420" }, { "peer": "f1e2872581de18bdc68060dc8edd3aa96368eb341e915aba86b450486b105a47", "balance": "-75990" } //... ] } ``` In Swarm, these per-peer balances represent trustful agreements between nodes. Tokens only actually change hands when a node settles a cheque. This can either be triggered manually or when a certain threshold is reached with a peer. In this case, a settlement takes place. You may view these using the settlements endpoint. More info can be found by using the chequebook API. ```bash curl localhost:1633/settlements| jq ``` ```json { "totalreceived": "718030", "totalsent": "0", "settlements": [ //... { "peer": "dce1833609db868e7611145b48224c061ea57fd14e784a278f2469f355292ca6", "received": "8987000000000", "sent": "0" } //... ] } ``` More information about the current received or sent cheques can also be found using the chequebook api. ```bash curl localhost:1633/chequebook/cheque | jq ``` ```json { "lastcheques": [ { "peer": "dce1833609db868e7611145b48224c061ea57fd14e784a278f2469f355292ca6", "lastreceived": { "beneficiary": "0x21b26864067deb88e2d5cdca512167815f2910d3", "chequebook": "0x4A373Db93ba54cab999e2C757bF5ca0356B42a3f", "payout": "8987000000000" }, "lastsent": null } //... ] } ``` As our node's participation in the network increases, we will begin to see more and more of these balances arriving. In the case that we have _received_ a settlement from another peer, we can ask our node to perform the relevant transactions on the blockchain, and cash our earnings out. To do this, we simply POST the relevant peer's address to the `cashout` endpoint. ```bash curl -X POST http://localhost:1633/chequebook/cashout/d7881307e793e389642ea733451db368c4c9b9e23f188cca659c8674d183a56b ``` ```json { "transactionHash": "0xba7b500e21fc0dc0d7163c13bb5fea235d4eb769d342e9c007f51ab8512a9a82" } ``` You may check the status of your transaction using the [xDAI Blockscout](https://gnosis.blockscout.com/). Finally, we can now see the status of the cashout transaction by sending a GET request to the same URL. ```bash curl http://localhost:1633/chequebook/cashout/d7881307e793e389642ea733451db368c4c9b9e23f188cca659c8674d183a56b | jq ``` ```json { "peer": "d7881307e793e389642ea733451db368c4c9b9e23f188cca659c8674d183a56b", "chequebook": "0xae315a9adf0920ba4f3353e2f011031ca701d247", "cumulativePayout": "179160", "beneficiary": "0x21b26864067deb88e2d5cdca512167815f2910d3", "transactionHash": "0xba7b500e21fc0dc0d7163c13bb5fea235d4eb769d342e9c007f51ab8512a9a82", "result": { "recipient": "0x312fe7fde9e0768337c9b3e3462189ea6f9f9066", "lastPayout": "179160", "bounced": false } } ``` Success, we earned our first xBZZ! 🐝 Now we have earned tokens, to withdraw our xBZZ from the chequebook contract back into our node's own wallet, we simply POST a request to the chequebook withdraw endpoint. ```bash curl -X POST http://localhost:1633/chequebook/withdraw\?amount\=1000 | jq ``` And conversely, if we have used more services than we have provided, we may deposit extra xBZZ into the chequebook contract by sending a POST request to the deposit endpoint. ```bash curl -X POST http://localhost:1633/chequebook/deposit\?amount\=1000 | jq ``` ```json { "transactionHash": "0xedc80ebc89e6d719e617a50c6900c3dd5dc2f283e1b8c447b9065d7c8280484a" } ``` You may then use [Blockscout](https://gnosis.blockscout.com/) to track your transaction and make sure it completed successfully. ## Managing uncashed cheques For the Bee process, the final step of earning xBZZ is cashing a cheque. It is worth noting that a cheque is not yet actual xBZZ. In Bee, a cheque, just like a real cheque, is a promise to hand over money upon request. In real life, you would present the cheque to a bank. In swarm life, we present the cheque to a smart-contract. Holding on to a swap-cheque is risky; it is possible that the owner of the chequebook has issued cheques worth more xBZZ than is contained in their chequebook contract. For this reason, it is important to cash out your cheques every so often. With the set of API endpoints, as offered by Bee, it is possible to develop a script that fully manages the uncashed cheques for you. As an example, we offer you a [very basic script](https://gist.github.com/ralph-pichler/3b5ccd7a5c5cd0500e6428752b37e975#file-cashout-sh), where you can manually cash out all cheques with a worth above a certain value. To use the script: 1. Download and save the script: ```bash wget -O cashout.sh https://gist.githubusercontent.com/ralph-pichler/3b5ccd7a5c5cd0500e6428752b37e975/raw/cashout.sh ``` 2. Make the file executable: ```bash chmod +x cashout.sh ``` 3. List all uncashed cheques and cash out your cheques above a certain value: - List: ```bash ./cashout.sh ``` :::info If running ./cashout.sh returns nothing, you currently have no uncashed cheques. ::: - Cashout all cheques: ```bash ./cashout.sh cashout-all ``` :::info Are you a Windows-user who is willing to help us? We are currently missing a simple cashout script for Windows. Please see the [issue](https://github.com/ethersphere/bee/issues/1092). ::: :::info You can find the officially deployed smart-contract by the Swarm team in the [swap-swear-and-swindle repository](https://github.com/ethersphere/swap-swear-and-swindle). ::: --- ## Configuration ## Configuration Methods and Priority There are three configuration methods, each with a different priority level. Configuration is processed in the following ascending order of preference: 1. Command Line Flags 2. Environment Variables 3. YAML Configuration File :::info All three methods may be used when running Bee using `bee start`. However when Bee is started as a service with tools like `systemctl` or `brew services`, only the YAML configuration file is supported by default. ::: ### Command Line Arguments Run `bee help printconfig` in your terminal to list the available command-line arguments and config option flags: ```bash Ethereum Swarm Bee Usage: bee [command] Available Commands: start Start a Swarm node init Initialise a Swarm node deploy Deploy and fund the chequebook contract version Print version number db Perform basic DB related operations split Split a file into chunks printconfig Print default or provided configuration in yaml format help Help about any command completion Generate the autocompletion script for the specified shell Flags: --config string config file (default is $HOME/.bee.yaml) -h, --help help for bee Use "bee [command] --help" for more information about a command. root@noah-bee:~# docker exec -it bee-1 bee help printconfig Print default or provided configuration in yaml format Usage: bee printconfig [flags] Flags: --allow-private-cidrs allow to advertise private CIDRs to the public network --api-addr string HTTP API listen address (default "127.0.0.1:1633") --autotls-ca-endpoint string autotls certificate authority endpoint (default "https://acme-v02.api.letsencrypt.org/directory") --autotls-domain string autotls domain (default "libp2p.direct") --autotls-registration-endpoint string autotls registration endpoint (default "https://registration.libp2p.direct") --block-sync-interval uint block number cache sync interval in blocks (default 10) --block-time uint chain block time (default 5) --blockchain-rpc-dial-timeout duration blockchain rpc TCP dial timeout (default 30s) --blockchain-rpc-endpoint string rpc blockchain endpoint --blockchain-rpc-idle-timeout duration blockchain rpc idle connection timeout (default 1m30s) --blockchain-rpc-keepalive duration blockchain rpc TCP keepalive interval (default 30s) --blockchain-rpc-tls-timeout duration blockchain rpc TLS handshake timeout (default 10s) --bootnode strings initial nodes to connect to (default [/dnsaddr/mainnet.ethswarm.org]) --bootnode-mode cause the node to always accept incoming connections --bzz-token-address string bzz token contract address --cache-capacity uint cache capacity in chunks, multiply by 4096 to get approximate capacity in bytes (default 1000000) --cache-retrieval enable forwarded content caching (default true) --chequebook-enable enable chequebook (default true) --chequebook-min-balance string minimum chequebook token balance required for verification, in token small units (default 11 BZZ) (default "110000000000000000") --chequebook-verification reject full-node hive/handshake records that carry no chequebook address --cors-allowed-origins strings origins with CORS headers enabled --data-dir string data directory (default "/home/bee/.bee") --db-block-cache-capacity uint size of block cache of the database in bytes (default 33554432) --db-disable-seeks-compaction disables db compactions triggered by seeks (default true) --db-open-files-limit uint number of open files allowed by database (default 200) --db-write-buffer-size uint size of the database write buffer in bytes (default 33554432) --full-node cause the node to start in full mode --gas-limit-fallback uint gas limit fallback when estimation fails for contract transactions (default 500000) -h, --help help for printconfig --mainnet triggers connect to main net bootnodes. (default true) --minimum-gas-tip-cap uint minimum gas tip cap in wei for transactions, 0 means use suggested gas tip cap --minimum-storage-radius uint minimum radius storage threshold --nat-addr string NAT exposed address --nat-wss-addr string WSS NAT exposed address --neighborhood-suggester string suggester for target neighborhood (default "https://api.swarmscan.io/v1/network/neighborhoods/suggestion") --network-id uint ID of the Swarm network (default 1) --p2p-addr string P2P listen address (default ":1634") --p2p-ws-enable enable P2P WebSocket transport --p2p-wss-addr string p2p wss address (default ":1635") --p2p-wss-enable Enable Secure WebSocket P2P connections --password string password for decrypting keys --password-file string path to a file that contains password for decrypting keys --payment-early-percent int percentage below the peers payment threshold when we initiate settlement (default 50) --payment-threshold string threshold in BZZ where you expect to get paid from your peers (default "13500000") --payment-tolerance-percent int excess debt above payment threshold in percentages where you disconnect from your peer (default 25) --postage-stamp-address string postage stamp contract address --postage-stamp-start-block uint postage stamp contract start block number --pprof-mutex enable pprof mutex profile --pprof-profile enable pprof block profile --price-oracle-address string price oracle contract address --redistribution-address string redistribution contract address --reserve-capacity-doubling int reserve capacity doubling --resolver-options strings ENS compatible API endpoint for a TLD and with contract address, can be repeated, format [tld:][contract-addr@]url --resync forces the node to resync postage contract data --skip-postage-snapshot skip postage snapshot --staking-address string staking contract address --statestore-cache-capacity uint lru memory caching capacity in number of statestore entries (default 100000) --static-nodes strings protect nodes from getting kicked out on bootnode --storage-incentives-enable enable storage incentives feature (default true) --swap-enable enable swap --swap-factory-address string swap factory addresses --swap-initial-deposit string initial deposit if deploying a new chequebook (default "0") --target-neighborhood string neighborhood to target in binary format (ex: 111111001) for mining the initial overlay --tracing-enable enable tracing --tracing-endpoint string endpoint to send tracing data (default "127.0.0.1:6831") --tracing-host string host to send tracing data --tracing-port string port to send tracing data --tracing-service-name string service name identifier for tracing (default "bee") --transaction-debug-mode skips the gas estimate step for contract transactions --use-simd-hashing use SIMD BMT hasher (available only on linux amd64 platforms) --verbosity string log verbosity level 0=silent, 1=error, 2=warn, 3=info, 4=debug, 5=trace (default "info") --warmup-time duration maximum node warmup duration; proceeds when stable or after this time (default 5m0s) --welcome-message string send a welcome message string during handshakes --withdrawal-addresses-whitelist strings withdrawal target addresses Global Flags: --config string config file (default is $HOME/.bee.yaml) ``` ### Environment variables Bee configuration can also be set using environment variables. Environment variables are set as variables in your operating system's session or systemd configuration file. To set an environment variable, type the following in your terminal session. ```bash export VARIABLE_NAME=variableValue ``` Verify that it is correctly set by running `echo $VARIABLE_NAME`. All available configuration options are available as `BEE` prefixed, capitalised, and underscored environment variables, e.g. `--api-addr` becomes `BEE_API_ADDR`. ### YAML configuration file You can view the default contents of the `bee.yaml` configuration file using the `bee printconfig` command: ```bash bee printconfig ``` ```yaml # allow to advertise private CIDRs to the public network allow-private-cidrs: false # HTTP API listen address api-addr: 127.0.0.1:1633 # autotls certificate authority endpoint autotls-ca-endpoint: https://acme-v02.api.letsencrypt.org/directory # autotls domain autotls-domain: libp2p.direct # autotls registration endpoint autotls-registration-endpoint: https://registration.libp2p.direct # block number cache sync interval in blocks block-sync-interval: "10" # chain block time block-time: "5" # blockchain rpc TCP dial timeout blockchain-rpc-dial-timeout: 30s # rpc blockchain endpoint blockchain-rpc-endpoint: "" # blockchain rpc idle connection timeout blockchain-rpc-idle-timeout: 1m30s # blockchain rpc TCP keepalive interval blockchain-rpc-keepalive: 30s # blockchain rpc TLS handshake timeout blockchain-rpc-tls-timeout: 10s # initial nodes to connect to bootnode: - /dnsaddr/mainnet.ethswarm.org # cause the node to always accept incoming connections bootnode-mode: false # bzz token contract address bzz-token-address: "" # cache capacity in chunks, multiply by 4096 to get approximate capacity in bytes cache-capacity: "1000000" # enable forwarded content caching cache-retrieval: true # enable chequebook chequebook-enable: true # minimum chequebook token balance required for verification, in token small units (default 11 BZZ) chequebook-min-balance: "110000000000000000" # reject full-node hive/handshake records that carry no chequebook address chequebook-verification: false # config file (default is $HOME/.bee.yaml) config: /home/bee/.bee.yaml # origins with CORS headers enabled cors-allowed-origins: [] # data directory data-dir: /home/bee/.bee # size of block cache of the database in bytes db-block-cache-capacity: "33554432" # disables db compactions triggered by seeks db-disable-seeks-compaction: true # number of open files allowed by database db-open-files-limit: "200" # size of the database write buffer in bytes db-write-buffer-size: "33554432" # cause the node to start in full mode full-node: false # gas limit fallback when estimation fails for contract transactions gas-limit-fallback: "500000" # help for printconfig help: false # triggers connect to main net bootnodes. mainnet: true # minimum gas tip cap in wei for transactions, 0 means use suggested gas tip cap minimum-gas-tip-cap: "0" # minimum radius storage threshold minimum-storage-radius: "0" # NAT exposed address nat-addr: "" # WSS NAT exposed address nat-wss-addr: "" # suggester for target neighborhood neighborhood-suggester: https://api.swarmscan.io/v1/network/neighborhoods/suggestion # ID of the Swarm network network-id: "1" # P2P listen address p2p-addr: :1634 # enable P2P WebSocket transport p2p-ws-enable: false # p2p wss address p2p-wss-addr: :1635 # Enable Secure WebSocket P2P connections p2p-wss-enable: false # password for decrypting keys password: "" # path to a file that contains password for decrypting keys password-file: "" # percentage below the peers payment threshold when we initiate settlement payment-early-percent: 50 # threshold in BZZ where you expect to get paid from your peers payment-threshold: "13500000" # excess debt above payment threshold in percentages where you disconnect from your peer payment-tolerance-percent: 25 # postage stamp contract address postage-stamp-address: "" # postage stamp contract start block number postage-stamp-start-block: "0" # enable pprof mutex profile pprof-mutex: false # enable pprof block profile pprof-profile: false # price oracle contract address price-oracle-address: "" # redistribution contract address redistribution-address: "" # reserve capacity doubling reserve-capacity-doubling: 0 # ENS compatible API endpoint for a TLD and with contract address, can be repeated, format [tld:][contract-addr@]url resolver-options: [] # forces the node to resync postage contract data resync: false # skip postage snapshot skip-postage-snapshot: false # staking contract address staking-address: "" # lru memory caching capacity in number of statestore entries statestore-cache-capacity: "100000" # protect nodes from getting kicked out on bootnode static-nodes: [] # enable storage incentives feature storage-incentives-enable: true # enable swap swap-enable: false # swap factory addresses swap-factory-address: "" # initial deposit if deploying a new chequebook swap-initial-deposit: "0" # neighborhood to target in binary format (ex: 111111001) for mining the initial overlay target-neighborhood: "" # enable tracing tracing-enable: false # endpoint to send tracing data tracing-endpoint: 127.0.0.1:6831 # host to send tracing data tracing-host: "" # port to send tracing data tracing-port: "" # service name identifier for tracing tracing-service-name: bee # skips the gas estimate step for contract transactions transaction-debug-mode: false # use SIMD BMT hasher (available only on linux amd64 platforms) use-simd-hashing: false # log verbosity level 0=silent, 1=error, 2=warn, 3=info, 4=debug, 5=trace verbosity: info # maximum node warmup duration; proceeds when stable or after this time warmup-time: 5m0s # send a welcome message string during handshakes welcome-message: "" # withdrawal target addresses withdrawal-addresses-whitelist: [] ``` :::info Note that depending on whether Bee is started directly with the `bee start` command or started as a service with `systemctl` / `brew services`, the default directory for the YAML configuration file (shown in the `config` option above) [will be different](./configuration.md). ::: To change your node's configuration, simply edit the YAML file and restart Bee: Open the config file for editing: ```bash sudo vi /etc/bee/bee.yaml ``` After saving your changes, restart your node: ```bash sudo systemctl restart bee ``` Open the config file for editing: ```bash sudo vi /opt/homebrew/etc/swarm-bee/bee.yaml ``` After saving your changes, restart your node: ```bash brew services restart swarm-bee ``` Open the config file for editing: ```bash sudo vi /usr/local/etc/swarm-bee/bee.yaml ``` After saving your changes, restart your node: ```bash brew services restart swarm-bee ``` ## Manually generating YAML config file for *bee start* No YAML file is generated during installation when using the [shell script install method](./../installation/shell-script.md), so you must generate one if you wish to use a YAML file to specify your configuration options. To do this you can use the `bee printconfig` command to print out a set of default options and save it to a new file in the default location: ```bash bee printconfig &> $HOME/.bee.yaml ``` :::info Note that `bee printconfig` prints the default configuration for your node, not the current configuration including any changes. ::: When using `bee.yaml` with the `bee start` command, make sure to use the `--config` flag to specify the location of your configuration file. ## Node Types There are three node types which each offer varying levels of functionality - ***full***, ***light***, and ***ultra-light***. You can configure your node to run as any of these three types by setting the related options within your configuration. For a deeper dive into each node type and its features and limitations, refer to the [Node Types](./node-types.md) page. ### How to Set Node Type There are three relevant options which are used to set your node type: `full-node`, `swap-enable`, and `blockchain-rpc-endpoint`. The required option values for each node type are outlined below: | Node Type | `full-node` | `swap-enable` | `blockchain-rpc-endpoint` | Functionality | | ---------------- | ----------- | ------------- | ------------------------- | ---------------------------------------------------------------------------------- | | Full Node | `true` | `true` | Required | Full functionality, including uploads, downloads, and Swarm network participation. | | Light Node | `false` | `true` | Required | Supports uploading and downloading only. | | Ultra-Light Node | `false` | `false` | Not required | Free-tier downloads only. | ## Configuration Examples Bee nodes can be configured using command-line flags, environment variables, or a YAML configuration file: ### Full Node Configuration #### Using Command-Line Arguments ```bash bee start \ --password mypassword \ --full-node \ --swap-enable \ --blockchain-rpc-endpoint https://xdai.fairdatasociety.org ``` #### Using Environment Variables ```bash export BEE_PASSWORD="mypassword" export BEE_FULL_NODE="true" export BEE_SWAP_ENABLE="true" export BEE_BLOCKCHAIN_RPC_ENDPOINT="https://xdai.fairdatasociety.org" bee start ``` #### Using YAML Configuration ```yaml password: mypassword full-node: true swap-enable: true blockchain-rpc-endpoint: "https://xdai.fairdatasociety.org" ``` ### Light Node Configuration #### Using Command-Line Arguments ```bash bee start \ --password mypassword \ --swap-enable \ --blockchain-rpc-endpoint https://xdai.fairdatasociety.org ``` #### Using Environment Variables ```bash export BEE_PASSWORD="mypassword" export BEE_SWAP_ENABLE="true" export BEE_BLOCKCHAIN_RPC_ENDPOINT="https://xdai.fairdatasociety.org" bee start ``` #### Using YAML Configuration ```yaml password: mypassword swap-enable: true blockchain-rpc-endpoint: "https://xdai.fairdatasociety.org" ``` ### Ultra-Light Node Configuration #### Using Command-Line Arguments ```bash bee start \ --password mypassword ``` #### Using Environment Variables ```bash export BEE_PASSWORD="mypassword" bee start ``` #### Using YAML Configuration ```yaml password: mypassword ``` ## Default Data and Config Directories Depending on the operating system and startup method used, the default directories for your node will differ: ### Bee Service Default Directories (Package Manager Install) When installed using a package manager, Bee is set up to run as a service with default data and configuration directories set up automatically during the installation. The examples below include default directories for Linux and macOS. You can find the complete details of default directories for different operating systems in the `bee.yaml` files included in the [packaging folder of the Bee repo](https://github.com/ethersphere/bee/tree/master/packaging). The default data folder and config file locations: ```yaml data-dir: /var/lib/bee config: /etc/bee/bee.yaml ``` The default data folder and config file locations: ```yaml data-dir: /opt/homebrew/var/lib/swarm-bee config: /opt/homebrew/etc/swarm-bee/bee.yaml ``` The default data folder and config file locations: ```yaml data-dir: /usr/local/var/lib/swarm-bee/ config: /usr/local/etc/swarm-bee/bee.yaml ``` ### Shell Script Install Default Directories For all operating systems, the default data and config directories for the `bee start` startup method can be found using the `bee printconfig` command: This will print out a complete default Bee node configuration file to the terminal, the `config` and `data-dir` values show the default directories for your system: ```yaml config: /root/.bee.yaml data-dir: /root/.bee ``` :::info The default directories for your system may differ from the example above, so make sure to run the `bee printconfig` command to view the default directories for your system. ::: ## Create Password A password is required for all modes, and can either be set directly in text through the `password` configuration option or alternatively a file can be used by setting the `password-file` option to the path where your password file is located. ## Setting Blockchain RPC endpoint :::warning A RPC endpoint for *a full archival Gnosis Chain node is required* since a Bee node must sync all data starting from when the [postage stamp smart contract was created](https://gnosisscan.io/tx/0x3427deb106b30a7d23f7ce9d2465f2d83945948c5aeddba55337c318fb56ec25). The free RPC endpoint offered by the Fair Data Society (https://xdai.fairdatasociety.org) will work since it is a full archival node, but running Bee with other public free RPC endpoints from non-archive nodes will result in the `storage: not found` error. If you do encounter the `storage: not found` error, update your RPC endpoint to one for a full archival node, and restart your node with the `resync` option set to `true`. ::: Full and light Bee nodes require a Gnosis Chain RPC endpoint in order to sync blockchain data and issue transactions (not required for ultra-light nodes). To set your RPC endpoint, specify it with the `blockchain-rpc-endpoint` value, which is set to an empty string by default. ```yaml # bee.yaml blockchain-rpc-endpoint: https://xdai.fairdatasociety.org ``` We recommend you [run your own Gnosis Chain node](https://docs.gnosischain.com/node/), but you may also consider using a paid RPC endpoint provider such as [GetBlock](https://getblock.io/). ### RPC Providers While we recommend running your own Gnosis Chain node for your RPC endpoint, you may wish to use a third party provider instead. For a comprehensive list of RPC providers, refer to the [Gnosis Chain documentation](https://docs.gnosischain.com/tools/RPC%20Providers/). The list includes both free and paid RPC providers (refer to [warning above](#setting-blockchain-rpc-endpoint) about free RPC providers). For running a light node or for testing out a single full node you can use the free RPC endpoint provided by the Fair Data Society: `https://xdai.fairdatasociety.org`. ### Block number sync interval Bee periodically reads the current block number from your blockchain RPC endpoint and estimates it locally in between those reads, which significantly reduces how often it calls the RPC. The `block-sync-interval` option controls how frequently the real block number is refreshed: ```yaml # bee.yaml block-sync-interval: 10 ``` - The default is `10`, so operators who want the default do not need to set anything. - A higher value means fewer RPC calls, at the cost of a slightly staler block-number estimate. - `1` refreshes the block number as often as possible. - `0` is not allowed; the node treats it as "no interval" and clamps it to `1`. The equivalent flag is `--block-sync-interval` and the environment variable is `BEE_BLOCK_SYNC_INTERVAL`. This is especially useful for operators on paid or rate-limited RPC providers who want to lower request volume. ## SIMD hashing (Optional) Bee can use a hardware-accelerated (SIMD) implementation of its chunk hasher, a frequent and CPU-intensive operation. On supported hardware this noticeably lowers the CPU cost of hashing, which speeds up uploads and other hashing-heavy work. This is **opt-in and off by default**. It is currently available **only on Linux x86-64 (amd64)**; on all other platforms (Windows, macOS, ARM) the node automatically falls back to the standard hasher, with no action required and no regression. To enable it, set the `use-simd-hashing` option: ```yaml # bee.yaml use-simd-hashing: true ``` The equivalent flag is `--use-simd-hashing` and the environment variable is `BEE_USE_SIMD_HASHING`. ## Configuring Swap Initial Deposit (Optional) When running your Bee node with SWAP enabled for the first time, your node will deploy a 'chequebook' contract using the canonical factory contract which is deployed by Swarm. Once the chequebook is deployed, Bee will (optionally) deposit a certain amount of xBZZ in the chequebook contract so that it can pay other nodes in return for their services. The amount of xBZZ transferred to the chequebook is set by the `swap-initial-deposit` configuration setting (it may be left at the default value of zero or commented out). ## Chequebook Verification (Optional) Full node operators can optionally require that incoming peer connections come from nodes that maintain a minimum chequebook balance. When enabled, the node checks the chequebook balance of each incoming full-node peer and rejects connections from peers whose balance falls below the configured threshold. This feature is **disabled by default** and only applies to full nodes with chequebook and chain functionality enabled. Light nodes skip chequebook verification regardless of configuration. Remote peers do not need to enable chequebook verification themselves in order to be accepted — only the verifying node needs to have it enabled. To enable chequebook verification, set `chequebook-verification` to `true`: ```yaml # bee.yaml chequebook-verification: true ``` The default minimum balance threshold is **11 BZZ**. The value is set in the token's smallest unit (PLUR), where 1 BZZ = 10^16 PLUR, so the default is `110000000000000000`. To configure a different threshold, set `chequebook-min-balance` accordingly: ```yaml # bee.yaml chequebook-verification: true chequebook-min-balance: "110000000000000000" # 11 BZZ, expressed in PLUR ``` Or using command-line flags: ```bash bee start \ --chequebook-verification \ --chequebook-min-balance 110000000000000000 ``` :::info Chequebook verification is an optional defense layer for full node operators who want to avoid connecting to peers that do not maintain a sufficient chequebook balance. It was introduced in Bee v2.8.0. ::: ## NAT address Swarm is all about sharing and storing chunks of data. To enable other Bees (also known as _peers_) to connect to your Bee, you must broadcast your public IP address in order to ensure that Bee is reachable on the correct p2p port (default `1634`). We recommend that you [manually configure your external IP and check connectivity](/docs/bee/installation/connectivity) to ensure your Bee is able to receive connections from other peers. First, determine your public IP address: ```bash curl icanhazip.com ``` ```bash 123.123.123.123 ``` Then configure your node, including your p2p port (default 1634). ```yaml # bee.yaml nat-addr: "123.123.123.123:1634" ``` Ensure `nat-addr` and `nat-wss-addr` if used are set to valid `host:port` values — invalid values prevent the node from starting. ## ENS Resolution (Optional) The [ENS](https://ens.domains/) domain resolution system is used to host websites on Bee, and in order to use this your Bee must be connected to a mainnet Ethereum blockchain node. We recommend you run your own ethereum node. An option for resource restricted devices is geth+nimbus and a guide can be found [here](https://ethereum-on-arm-documentation.readthedocs.io/en/latest/). Other options include [dappnode](https://dappnode.com/), [nicenode](https://www.nicenode.xyz/), [stereum](https://stereum.net/) and [avado](https://ava.do/). If you do not wish to run your own Ethereum node, you may use a blockchain RPC service provider such as [Infura](https://infura.io). After signing up for Infura, simply set your `--resolver-options` to `https://mainnet.infura.io/v3/your-api-key`. ```yaml # bee.yaml resolver-options: ["https://mainnet.infura.io/v3/<>"] ``` ## Sepolia Testnet Configuration In order to operate a Bee node on the Sepolia testnet, you need to change `mainnet` to `false`, and provide a valid Sepolia testnet RPC endpoint through the `blockchain-rpc-endpoint` option. Here is an example of a full configuration for a testnet full node: ```yaml data-dir: /home/username/bee/sepolia # Specified an alternate "data-dir" for our testnet node data full-node: true mainnet: false # Changed to "false" password: password blockchain-rpc-endpoint: wss://sepolia.infura.io/ws/v3/ # Replaced the Gnosis Chain RPC with a Sepolia testnet RPC endpoint swap-enable: true verbosity: 5 welcome-message: "welcome-from-the-hive" warmup-time: 30s ``` ### Funding Testnet Node Make sure to fund your node with Sepolia ETH rather than xDAI to pay for gas on the Sepolia testnet. There are many public faucets you can use to obtain Sepolia ETH, such as [this one from Infura](https://www.infura.io/faucet/sepolia). To get Sepolia BZZ (sBZZ) you can use [this Uniswap market](https://app.uniswap.org/swap?outputCurrency=0x543dDb01Ba47acB11de34891cD86B675F04840db&inputCurrency=ETH), just make sure that you've switched to the Sepolia network in your browser wallet. --- ## Introduction(Working-with-bee) In this section we cover everything a node operator needs to know about working with Bee: ## Configuration Learn how to [configure your node](./configuration.md), and the details behind all the configuration options Bee provides. ## Bee API Access the HTTP API directly for [detailed information about your Bee](./bee-api.md). ## Logs and Files Find out where Bee stores your [logs and files](./logs-and-files.md). ## Swarm CLI You can use the [`swarm-cli`command line tool](./swarm-cli.md) to monitor your Bee's status, cash out your cheques, upload data to the swarm and more! ## Cashing Out Get your cheques cashed and bank your xBZZ. [See this guide](./cashing-out.md) to receiving payments from your peers. ## Monitoring and Metrics There is a lot going on inside Bee, we provide tools and metrics to help you [find out what's going on](./monitoring.md). ## Backups [Keep your important data safe](./backups.md), Bee stores important state and key information on your hardrive, make sure you keep a secure copy in case of disaster. ## Upgrading Find out how to [keep your Bee up to date](./upgrade.md) with the latest and greatest releases, and make sure you're tuned into our release announcements. ## Uninstalling Bee We hope you won't need to remove Bee. If you do, please let us know if you had issues so we can help resolve them for our beloved network. Here's the guide to [removing Bee from your system](./uninstalling-bee.md). --- ## Logging in Bee This section provides an overview of logging in Bee, including log locations, exporting logs, managing verbosity levels, and using fine-grained control for specific loggers. :::info Bee uses a structured logging format compatible with popular tools such as [Grafana](https://grafana.com/) and [Elasticsearch](https://www.elastic.co/elasticsearch). Structured logging helps streamline log analysis and management by organizing data into machine-readable formats, enabling easy integration with monitoring and debugging tools. ::: :::warning Bee logs can be verbose by default, potentially consuming significant disk space over time. Consider implementing [log rotation](https://en.wikipedia.org/wiki/Log_rotation) to prevent excessive disk utilization. ::: ## Log Locations ### **Linux (Package Manager Installation)** When installed via a package manager (e.g., `APT`, `RPM`), Bee runs as a **systemd service**, and logs are managed by the system journal, **journalctl**. View logs with: ```bash journalctl --lines=100 --follow --unit bee ``` Export all logs as JSON: ```bash journalctl --unit bee --output=json > bee-logs.json ``` Export logs for a specific time range: ```bash journalctl --since "1 hour ago" --output=json --unit bee > bee-logs.json ``` Learn more about `journalctl` usage and filtering logs in this [tutorial](https://www.digitalocean.com/community/tutorials/how-to-use-journalctl-to-view-and-manipulate-systemd-logs) from DigitalOcean. ### **macOS (Homebrew Installation)** For a Homebrew installation on macOS, logs are saved to: ```bash /usr/local/var/log/swarm-bee/bee.log ``` View logs in real-time: ```bash tail -f /usr/local/var/log/swarm-bee/bee.log ``` ### **Docker** Docker saves **stdout** and **stderr** output as JSON files by default. Logs are stored in: ``` /var/lib/docker/containers//-json.log ``` View logs in real time: ```bash docker logs -f ``` Export logs to a file: ```bash docker logs > bee-logs.json ``` Export logs for a specific time range: ```bash docker logs --since "30m" > bee-logs.json ``` See [Docker documentation](https://docs.docker.com/reference/cli/docker/container/logs/) for additional options. ### **Shell Script** For a shell script-installed Bee started using `bee start`, logs are sent to **stdout** and **stderr** by default, which means they will appear in the terminal. They are **not saved to disk by default**. To save logs to a file, redirect **stdout** and **stderr**: ```bash bee start --password > bee.log 2>&1 & ``` View recent logs and follow for updates: ```bash tail -f bee.log ``` ## Logging Levels Bee supports the following log levels: | Level | Description | |-------------|------------------------------------| | `0=silent` | No logs. | | `1=error` | Critical errors only. | | `2=warn` | Warnings and errors. | | `3=info` | General operational logs (default).| | `4=debug` | Detailed diagnostic logs. | | `5=trace` | Highly granular logs for debugging.| ### Behavior of Log Levels Log levels are cumulative: setting a higher verbosity includes all lower levels. For example, `debug` will output logs at `debug`, `info`, `warn`, and `error` levels. ## Setting Verbosity The general verbosity level can be set using the `verbosity` configuration option in order to display all log messages up to the selected level of verbosity. ### **YAML Config File** Set the `verbosity` parameter in `config.yaml`: ```yaml # Log verbosity: 0=silent, 1=error, 2=warn, 3=info, 4=debug, 5=trace verbosity: debug ``` ### **Command Line Flag** Set the verbosity level (0-5) when starting Bee: ```bash bee start --verbosity debug ``` ### **Environment Variable** Set `BEE_VERBOSITY` before starting Bee: ```bash export BEE_VERBOSITY=debug bee start ``` ## Fine-Grained Logging Control Bee allows fine-grained control of logging levels for specific subsystems using the **`/loggers` API endpoint**. This enables adjustments without restarting the node. ### **1. Retrieving Loggers List** Retrieve a list of active loggers and their verbosity levels: ```bash curl http://localhost:1633/loggers | jq ``` The list of loggers includes detailed entries for each subsystem. Below is an example for the `node/api` logger: ```json { "logger": "node/api", "verbosity": "info", "subsystem": "node/api[0][]>>824634474528", "id": "bm9kZS9hcGlbMF1bXT4-ODI0NjM0NDc0NTI4" } ``` - **`id`**: The Base64-encoded identifier used to adjust the logger’s verbosity. - **`verbosity`**: The current log level. ### **2. Adjusting Logger Verbosity** You can dynamically adjust the log level for any logger without restarting Bee. **Syntax**: ```bash curl -X PUT http://localhost:1633/loggers// ``` - **``**: The Base64-encoded logger name retrieved from `/loggers`. - **``**: Desired log level (`none`, `error`, `warn`, `info`, `debug`, `trace`). **Example**: Set `node/api` to `debug`: ```bash curl -X PUT http://localhost:1633/loggers/bm9kZS9hcGlbMF1bXT4-ODI0NjM0NDc0NTI4/debug ``` ### Log Level Behavior Note Log levels are cumulative. When a logger is set to a specific level, it will include all log messages at that level and below. For example: - Setting a logger to `info` will show logs at `info`, `warn`, and `error`. - Logs at higher levels (`debug` and `trace`) will **not** be displayed. --- ## Monitoring Your Node Bee nodes expose runtime metrics in Prometheus format, which you can collect and visualise with Grafana to understand what your Bee has been up to. Navigate to `http://localhost:1633/metrics`. The /metrics page shows a snapshot of your Bee node's metrics at the moment you load it. To make these raw metrics useful, you need to record them over time. To record Bee's metrics over time, we will use [Prometheus](https://prometheus.io/docs/introduction/overview/). Simply install, configure as follows, and restart! For Ubuntu and other Debian based Linux distributions install using `apt`: ```bash sudo apt install prometheus ``` And configure `localhost:1633` as a `target` in the `static_configs`. ```bash sudo vim /etc/prometheus/prometheus.yml ``` ```yaml static_configs: - targets: ["localhost:9090", "localhost:1633"] ``` Navigate to [http://localhost:9090](http://localhost:9090) to see the Prometheus user interface. Now that our metrics are being scraped into Prometheus' database, we can use it as a data source which is used by [Grafana](https://grafana.com/oss/grafana/) to display the metrics as a time series graph on the dashboard. Type `bee_` in the 'expression' or 'metrics' field in Prometheus or Grafana respectively to see the list of metrics available. Here's a few to get you started! ``` rate(bee_swap_cheques_received[1d]) rate(bee_swap_cheques_sent[1d]) rate(bee_swap_cheques_rejected[1d]) ``` Share your creations in the [#node-operators](https://discord.gg/kHRyMNpw7t) channel of our Discord server! --- ## Node Types Bee nodes can operate in three different modes depending on the user's needs, ranging from full-featured nodes that contribute storage to the network and earn incentives to simpler modes that only download and upload data. This guide outlines the three primary node types — **_Full_**, **_Light_**, and **_Ultra-Light_** — along with their configurations, capabilities, and limitations. All three modes can run on ordinary consumer computers, without requiring any extraordinary hardware. What differs between them is the feature set, how much disk space and bandwidth the node uses, and whether it needs a blockchain connection and funds. Choosing the right node type depends on your goals, whether it's participating in the Swarm network as a storage provider, developing applications that use Swarm's decentralized storage and messaging, or simply exploring the technology with minimal setup. ## What are the Bee node types? {#node-types-overview} Bee can operate in different modes, each tailored to specific use cases: | Feature | Full Node | Light Node | Ultra-Light Node | | ----------------------------------------------------------------------- | --------- | ---------- | ---------------- | | Free tier [downloads](./../../develop/upload-and-download.md) | ✅ | ✅ | ✅ | | [Uploading](./../../develop/upload-and-download.md) (Can purchase [postage stamp batches](./../../develop/tools-and-features/buy-a-stamp-batch.md)) | ✅ | ✅ | ❌ | | Can exceed free tier downloads | ✅ | ✅ | ❌ | | Storage sharing | ✅ | ❌ | ❌ | | [Storage incentives](./staking.md) | ✅ | ❌ | ❌ | | [Bandwidth incentives](./../../concepts/incentives/bandwidth-incentives.md) | ✅ | ❌ | ❌ | | [PSS messaging](./../../develop/tools-and-features/pss.md) | ✅ | ❌ | ❌ | ## What is a full node? {#full-node} Full nodes are the most feature-rich nodes in the Swarm network. They provide full upload and download capabilities, store and serve data, and participate in storage and bandwidth incentives. A full node uses more disk space and bandwidth than the lighter modes and needs a funded blockchain connection, but its CPU and memory requirements stay low enough for everyday consumer hardware. Full nodes are ideal for users who want to contribute to the Swarm network and earn incentives, as well as developers who require access to all Bee features including messaging features such as PSS and GSOC. ### Full node specifications A full node does not need powerful hardware. The requirements below are met by most laptops and desktops, and even single-board computers such as a [Raspberry Pi](https://en.wikipedia.org/wiki/Raspberry_Pi) with an attached SSD. Disk space and sustained bandwidth are the main differences from the lighter node types: - **Processor**: Recent 2 GHz dual-core (2+ cores). 4-cores is comfortable if you intend to take part in the redistribution game. - **RAM**: 500 MB. - **Storage**: 20~30 GB SSD, ideally NVMe (HDD not recommended). - **Internet**: High-speed and stable connection. :::info Staking means taking part in the redistribution game, which raises CPU demand and disk I/O. This calls for more than 2 processor cores and SSD storage rather than an HDD. Before staking, test your setup using [the `/rchash` endpoint](./bee-api.md#rchash) to confirm your node can complete a sample in time. ::: A full node must also be connected to Gnosis Chain and hold enough funds to cover its on-chain operations: - **RPC endpoint**: A connection to Gnosis Chain (see [setting the blockchain RPC endpoint](./configuration.md#setting-blockchain-rpc-endpoint)). - **xDAI**: Minimum 0.1 xDAI for Gnosis Chain gas fees. - **(optional) xBZZ for staking**: 10 xBZZ, required only to participate in [storage incentives](./staking.md). ### Full node configuration To run Bee as a full node, set: - `full-node: true` - `swap-enable: true` - `blockchain-rpc-endpoint` to a valid Gnosis Chain RPC URL **Key characteristics:** - Can upload and download data. - Can purchase and manage postage stamp batches in order to pay for uploading data. - Can share disk space with the network and store chunks from Swarm uploaders. - Can participate in the storage incentives system by sharing disk space for a chance to earn xBZZ. - Can participate in the bandwidth incentives system and earn xBZZ by forwarding chunks for other nodes. - Requires a Gnosis Chain RPC endpoint for blockchain connectivity. - Supports full PSS messaging and GSOC. ## What is a light node? {#light-node} Light nodes provide a balance between functionality and resource efficiency. They can upload and download data but do not participate in chunk forwarding or storage for other nodes. Light nodes are suited for users who want to interact with Swarm without contributing storage to the network or maintaining a reserve. They can serve the needs of developers who need to access Swarm's download / upload features but do not need advanced messaging features such as PSS and GSOC which are available only in full nodes. Light node operators cannot earn xBZZ by participating in Swarm's incentives systems, as they do not participate in chunk forwarding or storage but only consume services, paying xBZZ for downloading data from full nodes and buying postage stamp batches for uploading data. :::info Light nodes do not benefit from plausible deniability when requesting data from the network. They are always the originator of requests. ::: ### Light node specifications No specific hardware is required to run a light node. It can run well on practically any commercially available computer released in recent years, including lightweight single-board computers such as [Raspberry Pi](https://en.wikipedia.org/wiki/Raspberry_Pi). Your downloads / uploads may be limited by your network speed, however, so if you plan on interacting extensively with the Swarm network, you should take your connection speed into consideration. ### Light node configuration To run Bee as a light node, set: - `full-node: false` - `swap-enable: true` - `blockchain-rpc-endpoint` to a valid Gnosis Chain RPC URL **Key characteristics:** - Can upload and download data. - Can purchase and manage postage stamp batches in order to pay for uploading data. - Requires a Gnosis Chain RPC endpoint for blockchain connectivity. **Limitations:** - Cannot share disk space with the network and store chunks from Swarm uploaders. - Cannot earn xBZZ by staking xBZZ and participating in the storage incentive system. - Cannot earn xBZZ by participating in the bandwidth incentives system. - Can send PSS messages but ***cannot*** receive them. - Can send outgoing GSOC updates but ***cannot*** receive them. ## What is an ultra-light node? {#ultra-light-node} Ultra-light nodes allow users to run a node without requiring a blockchain RPC endpoint. These nodes can download data within the free consumption threshold set by full nodes (this threshold may vary since it is [configurable](./configuration.md) by full node operators using the `payment-tolerance-percent` and `payment-threshold` options). Ultra-light nodes are designed for users who want to access the Swarm network with minimal resource requirements. These nodes can download data within the free consumption threshold but do not support uploads and cannot earn xBZZ by participating in Swarm's incentives systems. As with light nodes, your node's download speed will be limited by your network speed (however this may be less important of a consideration given that an ultra-light node is restricted to downloading within free tier limits anyway). :::warning As with light nodes, ultra-light nodes do not benefit from plausible deniability when requesting data from the network. When running without a blockchain connection, [bandwidth incentive payments (SWAP)](./../../concepts/incentives/bandwidth-incentives.md) cannot be made, increasing the risk of being blocklisted by other peers for exceeding their free-tier download limits. ::: ### Ultra-light node specifications As with the light node, there are no specific requirements to run an ultra-light node, and it will run on practically any commercially available hardware from recent years. ### Ultra-light node configuration Bee will start in ultra-light mode by default, but in order to explicitly configure your node to run as an ultra-light node, use the following options: - Set `full-node: false` - Set `blockchain-rpc-endpoint` to an empty string "" (or comment it out / remove it). - Set `swap-enable: false` **Key characteristics:** - Can download limited amounts of data. **Limitations:** - Cannot upload data. - Cannot purchase postage stamps. - Cannot share disk space with the network and store chunks from Swarm uploaders. - Cannot earn xBZZ by staking xBZZ and participating in the storage incentive system. - Cannot earn xBZZ by participating in the bandwidth incentives system. - Cannot use PSS or GSOC for sending or receiving. --- ## Staking Staking locks up xBZZ so your full node can join the **redistribution game** and earn a share of the network's storage-rent rewards. Staking requires a fully synced full node and a minimum of 10 xBZZ, and the stake is non-refundable. ## Quickstart Guide This guide will walk you through **staking xBZZ** and participating in the **redistribution game** to earn storage incentives. :::warning Staking requires a fully synced full node and a minimum of 10 xBZZ. See detailed [staking requirements](./staking.md#requirements) below. ::: ### Prerequisites - A small amount of xDAI to pay transaction fees - ~0.01 xDAI is enough to start - At least 10 xBZZ to deposit as non-refundable stake - A fully synced full Bee node :::tip If you don't already have xDAI or xBZZ, you will need to [get some](./../installation/fund-your-node.md#getting-tokens). ::: ### Step 1: Fund Your Node with xDAI and xBZZ Your node needs **xDAI** to pay for transaction fees on Gnosis Chain, and also needs **xBZZ** to deposit as stake. First, find your node's address using: ```bash swarm-cli addresses ``` This will print your node's various addresses. The one you need to fund is `Ethereum` :::tip The `Ethereum` term here refers to an Ethereum style address on Gnosis Chain. Do not send funds to the address on the Ethereum chain itself. ::: ```bash Node Addresses ------------------------------------------------------------------------------------------------------------------ Ethereum: 9a73f283cd9211b96b5ec63f7a81a0ddc847cd93 ... ``` Then, use the following command to check how much is required: ```bash swarm-cli status ``` At the bottom of the results printed to the terminal you will find the `Redistribution` section. From there you will see the `Minimum gas funds` item. That value is the minimum amount required to participate in a *single redistribution round*. :::tip If you plan on operating your node for an extended period, you will want to deposit quite a bit more than the minimum. You can start with **0.01 xDAI** to cover fees for the next few weeks/months of active staking, and then monitor actual usage and top-up when needed. ::: ``` Redistribution Reward: 0.0000000000000000 Has sufficient funds: true Fully synced: true Frozen: false Last selected round: 263526 Last played round: 0 Last won round: 0 Minimum gas funds: 0.000000000326250000 ``` Finally, send the required xDAI and xBZZ to the address you got from `swarm-cli addresses`. You will need to send at least 10 xBZZ to get started staking. :::tip Send **20 xBZZ** if using the [reserve doubling](#reserve-doubling) feature. ::: ### Step 2: Stake xBZZ Once your node has xDAI, stake **at least 10 xBZZ** (this is non-refundable). You can use the following `swarm-cli` command to stake 10 xBZZ: ```bash swarm-cli stake deposit --bzz 10 ``` After a moment, the staking transaction will complete. Then you can check that the transaction was successful: ```bash swarm-cli stake status ``` ```bash Staked xBZZ: 10 ``` :::tip **Optional:** Stake **20 xBZZ** if using the [reserve doubling](#reserve-doubling) feature. ::: ### Step 3: Check Status After staking you should [check your node's status](./staking.md#check-status) to make sure it is fully synced, fully funded, and operating properly. ### Step 4: Monitor & Maximize Rewards ✅ Make sure you are using a stable Gnosis Chain [RPC endpoint](./configuration.md#setting-blockchain-rpc-endpoint). ✅ [Check your node's status](./staking.md#check-status) to ensure it's operating properly. ✅ [Check `/rchash`](./bee-api.md#rchash) to ensure your node's performance is sufficient. ## Staking Overview To earn storage incentives by participating in the [redistribution game](./../../concepts/incentives/redistribution-game.md), full nodes must first deposit a minimum of 10 xBZZ as ***non-refundable*** stake. xDAI is also required to pay for ongoing Gnosis Chain transactions related to the redistribution game. :::danger Only stake your xBZZ if you intend to participate as a full node, as withdrawals are not possible. ::: ### Requirements - A [full node](./node-types.md) - see full node [recommend specs](./node-types.md#full-node-specifications). - A [high-performance RPC endpoint](./configuration.md#setting-blockchain-rpc-endpoint) connection to Gnosis Chain. - A minimum of 10 xBZZ to be used as ***non-refundable*** stake (the requirement is increased if [reserve doubling](./staking.md#reserve-doubling) is used). ### Check Status Use the `/redistributionstate` endpoint of the API to get more information about the redistribution status of the node. ```bash curl -X GET http://localhost:1633/redistributionstate | jq ``` ```bash { "minimumFunds": "18750000000000000", "hasSufficientFunds": true, "isFrozen": false, "isFullySynced": true, "phase": "commit", "round": 176319, "lastWonRound": 176024, "lastPlayedRound": 176182, "lastFrozenRound": 0, "block": 26800488, "reward": "10479124611072000", "fees": "30166618102500000" } ``` * `"minimumFunds": ` - The minimum xDAI needed to play a single round of the redistribution game (the unit is 1e-18 xDAI). * `"hasSufficientFunds": ` - Shows whether the node has enough xDAI balance to submit at least five storage incentives redistribution related transactions. If `false` the node will not be permitted to participate in next round. * `"isFrozen": ` - Shows node frozen status. * `"isFullySynced": ` - Shows whether node's localstore has completed full historical syncing with all connected peers. * `"phase": ` - Current phase of [redistribution game](./../../concepts/incentives/redistribution-game.md) (`commit`, `reveal`, or `claim`). * `"round": ` - Current round of redistribution game. The round number is determined by dividing the current Gnosis Chain block height by the number of blocks in one round. One round takes 152 blocks, so using the "block" output from the example above we can confirm that the round number is 176319 (block 26800488 / 152 blocks = round 176319). * `"lastWonRound": ` - Number of round last won by this node. * `"lastPlayedRound": ` - Number of the last round where node's neighborhood was selected to participate in redistribution game. * `"lastFrozenRound": ` The number the round when node was last frozen. * `"block": ` - Gnosis block of the current redistribution game. * `"reward": ` - Record of total reward received in [PLUR](./../../references/glossary.md#plur). * `"fees": ` - Record of total spent in 1E-18 xDAI on all redistribution related transactions. :::warning Do not shut down or update your node during an active redistribution round as it may cause them to lose out on winnings or become frozen. To see if your node is playing the current round, check if `lastPlayedRound` equals `round` in the output from the [`/redistributionstate` endpoint](/api/#tag/RedistributionState/paths/~1redistributionstate/get). ::: You should also check the [`/status`](/api/#tag/Node-Status/paths/~1status/get) endpoint: ```bash curl -s http://localhost:1633/status | jq ``` ```bash { "peer": "da7e5cc3ed9a46b6e7491d3bf738535d98112641380cbed2e9ddfe4cf4fc01c4", "proximity": 0, "beeMode": "full", "reserveSize": 3747532, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 183, "neighborhoodSize": 12, "batchCommitment": 133828050944, "isReachable": true } ``` **Expected values for a healthy staking node:** * `"beeMode": full` * `"pullsyncRate": 0` * `"isReachable": true` :::info If your node is not operating properly such as getting frozen or not participating in any rounds, see the [troubleshooting section](#troubleshooting). ::: ## Partial Stake Withdrawals If the price of xBZZ rises significantly and provides excess collateral, a partial withdrawal will be allowed down to the minimum required stake: ### Check for withdrawable stake ```bash curl http://localhost:1633/stake/withdrawable | jq ``` If there is any stake available for withdrawal, the amount will be displayed in PLUR: ```bash { "withdrawableStake": "18411" } ``` ### Withdraw available stake If there is any stake available for withdrawal, you can withdraw it using the `DELETE` method on `/stake/withdrawable`: ```bash curl -X DELETE http://localhost:1633/stake/withdrawable ``` ## Reserve Doubling The reserve doubling feature enables nodes to store chunks from a neighboring "sister" area, effectively increasing their reserve capacity twofold. By maintaining chunks from this sister neighborhood, a node becomes eligible to join the redistribution game whenever the sister neighborhood is chosen, effectively doubling its chances of participating. Although reserve doubling demands twice the disk storage and increases bandwidth usage for chunk syncing (with no additional bandwidth needed for chunk forwarding), its effect on CPU and RAM consumption remains minimal. This feature provides node operators with greater flexibility to optimize their nodes, aiming to achieve a higher reward-to-resource usage ratio. ### Step by Step Guide In order to double a node's reserve which has previously been operating without doubling, the `reserve-capacity-doubling` option must be updated from the default of `0` to `1` and restarted. There is also an increase in the xBZZ stake requirement from the minimum of 10 xBZZ to 20 xBZZ. #### **Step 1**: Set `reserve-capacity-doubling` to `1`. The reserve doubling feature can be enabled by setting the new `reserve-capacity-doubling` config option to `1` using the [configuration method](./configuration.md#configuration-methods-and-priority) of your choice. #### **Step 2**: Stake at least 20 xBZZ For doubling the reserve of a node which was previously operating which already has 10 xBZZ staked, simply stake an additional 10 xBZZ for a total of 20 xBZZ stake: :::info As always, ensure you properly convert the stake parameter to PLUR, where 1 PLUR equals 1e-16 xBZZ. ::: ```bash curl -X POST localhost:1633/stake/100000000000000000 ``` Or for a new node with zero staked xBZZ, the entire 20 xBZZ can be staked at once: ```bash curl -X POST localhost:1633/stake/200000000000000000 ``` We can use the `GET /stake` endpoint to confirm the total stake for our node: ```bash curl -s http://localhost:1633/stake | jq ``` ```bash { "stakedAmount": "200000000000000000" } ``` #### **Step 3**: Restart node After ensuring the node has at least 20 xBZZ staked and the `reserve-capacity-doubling` option has been set to `1`, restart the node. After restarting your node, it should then begin syncing chunks from its sister neighborhood. The `/status/neighborhoods` endpoint can be used to confirm that the node has doubled its reserve and is now syncing with its sister neighborhood: ```bash { "neighborhoods": [ { "neighborhood": "01111101011", "reserveSizeWithinRadius": 1148351, "proximity": 10 }, { "neighborhood": "01111101010", "reserveSizeWithinRadius": 1147423, "proximity": 11 } ] } ``` The output should list both your original and sister neighborhood. We can also check the `/status` endpoint to confirm our node is syncing new chunks: ```bash curl -s http://localhost:1633/status | jq ``` ```bash { "overlay": "be177e61b13b1caa20690311a909bd674a3c1ef5f00d60414f261856a8ad5c30", "proximity": 256, "beeMode": "full", "reserveSize": 4192792, "reserveSizeWithinRadius": 2295023, "pullsyncRate": 1.3033333333333332, "storageRadius": 10, "connectedPeers": 18, "neighborhoodSize": 1, "batchCommitment": 388104192, "isReachable": true, "lastSyncedBlock": 6982430 } ``` We can see that the `pullsyncRate` value is above zero, meaning that our node is currently syncing chunks, as expected. ## Maximize rewards There are two main factors which determine the chances for a staking node to win a reward — neighborhood selection and stake density. Both of these should be considered together before starting up a Bee node for the first time. See the [incentives page](./../../concepts/incentives/redistribution-game.md) for more context. ### Neighborhood selection By default when running a Bee node for the first time the node will use the [neighborhood suggestion tool](https://api.swarmscan.io/v1/network/neighborhoods/suggestion) from Swarmscan to find an optimal [neighborhood](./../../concepts/DISC/neighborhoods.md). While it is possible to manually choose a neighborhood using the `target-neighborhood` config option, we recommend not to do so as the suggestion tool will pick neighborhoods in order to maximize node earnings and network health. [Learn more](./../installation/set-target-neighborhood.md). ### Stake density Stake density is defined as: $$ \text{stake density} = \text{staked xBZZ} \times {2}^\text{storageDepth} $$ *To learn more about stake density and the mechanics of the incentives system, see the [incentives page](./../../concepts/incentives/redistribution-game.md).* Stake density determines the weighted chances of nodes within a neighborhood of winning rewards. The chance of winning within a neighborhood corresponds to stake density. Stake density can be increased by depositing more xBZZ as stake (note that stake withdrawals are not currently possible, so any staked xBZZ is not currently recoverable). Generally speaking, the minimum required stake of 10 xBZZ is sufficient, and rewards can be better maximized by operating more nodes over a greater range of neighborhoods rather than increasing stake. However this may not be true for all node operators depending on how many different neighborhoods they operate nodes in, and it also may change as network dynamics continue to evolve (join the `#node-operators` [Discord channel](https://discord.com/channels/799027393297514537/811553590170353685) to stay up to date with the latest discussions about staking and network dynamics). ### Chequebook verification Full node operators can optionally enable chequebook verification to reject incoming peers whose chequebook balance falls below a configurable threshold. This can help filter out poorly funded peers from your node's connections. Chequebook verification is disabled by default and can be enabled with the `--chequebook-verification` flag. The default minimum balance threshold is 11 BZZ and can be adjusted with `--chequebook-min-balance`. See the [configuration page](./configuration.md#chequebook-verification-optional) for details. ## Neighborhood Hopping :::warning There is a 2 round delay (with 152 Gnosis Chain blocks per redistribution game round) every time a node's neighborhood or stake is changed before it can participate in the redistribution game, moreover a node must fully sync the chunks from its new neighborhood before it can participate in the redistribution game, so hopping too frequently is not advised. ::: You can use the config option `target-neighborhood` to switch your node over to a new neighborhood. You may wish to use this option if your node's neighborhood becomes overpopulated. ### Checking neighborhood population For a quick check of your node's neighborhood population, we can use the `/status` endpoint: ```bash curl -s http://localhost:1633/status | jq { "peer": "e7b5c1aac67693268fdec98d097a8ccee1aabcf58e26c4512ea888256d0e6dff", "proximity": 0, "beeMode": "full", "reserveSize": 1055543, "reserveSizeWithinRadius": 1039749, "pullsyncRate": 42.67013868148148, "storageRadius": 11, "connectedPeers": 140, "neighborhoodSize": 6, "batchCommitment": 74463051776, "isReachable": false } ``` Here we can see that at the current `storageRadius` of 11, our node is in a neighborhood with size 6 from the `neighborhoodSize` value. Using the [Swarmscan neighborhoods tool](https://swarmscan.io/neighborhoods) we can see there are many neighborhoods with fewer nodes, so it would benefit us to move to less populated neighborhood: ![](/img/staking-swarmscan.png) While you might be tempted to simply pick one of these less populated neighborhoods, it is best practice to use the neighborhood suggester API instead, since it will help to prevent too many node operators rapidly moving to the same underpopulated neighborhoods, and also since the suggester takes a look at the next depth down to make sure that even in case of a neighborhood split, your node will end up in the smaller neighborhood. ```bash curl -s https://api.swarmscan.io/v1/network/neighborhoods/suggestion ``` Copy the binary number returned from the API: ```bash {"neighborhood":"01100011110"} ``` Use the binary number you just copied and set it as a string value for the `target-neighborhood` option in your config. ```bash # bee.yaml target-neighborhood: "01100011110" ``` ## Stake Migration If a new Bee release includes an updated staking contract, then you will be required to migrate your node's stake in order to continue normal operation. The stake migration process consists of the following steps: 1. Withdraw xBZZ 2. Stop node 3. Update and restart 4. Re-stake to the new contract ### Step 1: Withdraw xBZZ When a new version of Bee is released with an updated staking contract, the previous staking contract will be disabled, and stake withdrawals will be enabled. Once the contract is disabled, stake can be withdrawn by calling the `/stake` endpoint with the `DELETE` method: ```bash curl -X DELETE http://localhost:1633/stake ``` This command will withdraw all stake from the node to the node’s Gnosis Chain address. Confirm that the stake was withdrawn: ```bash curl -s http://localhost:1633/stake | jq ``` The value for `stakedAmount` should now be zero: ``` { "stakedAmount": "0" } ``` ### Step 2: Stop node This step will vary depending on how the node was set up: ```bash sudo systemctl stop bee ``` or ``` docker compose down ``` or ``` docker stop ``` etc. ### Step 3: Update and restart :::danger Before every Bee client upgrade, it is best practice to ALWAYS take a full [backup](./backups.md) of your node. ::: After withdrawing stake and stopping the node, update to the newest version of Bee. After updating, restart the node. You can use the `/health` endpoint to confirm your current Bee version: ```bash curl -s http://localhost:1633/health | jq ``` To confirm a successful update, check that the value for the `"version"` field in the results corresponds to the version number of the [latest](https://github.com/ethersphere/bee/releases/latest) Bee release. For example, if the latest version was 2.8.1, it would look like this: ```json { "status": "ok", "version": "2.8.1-7cf53193", "apiVersion": "8.1.0" } ``` *Make sure to check the [latest](https://github.com/ethersphere/bee/releases/latest) version number yourself, as the versions shown in examples in this guide may not always be up to date with the latest.* ### Step 4: Re-stake xBZZ After upgrading to the latest version and restarting, xBZZ should be re-staked into the new staking contract so that the node can continue to participate in the redistribution game. To stake the minimum required 10 xBZZ: :::tip Make sure to modify to the correct staking amount in case your node is using [reserve doubling](./staking.md#reserve-doubling). ::: ```bash curl -X POST localhost:1633/stake/100000000000000000 ``` Confirm that the staking transaction was successful: ```bash curl -s http://localhost:1633/stake | jq ``` The expected output after staking the minimum of 10 xBZZ: ```bash { "stakedAmount": "100000000000000000" } ``` Congratulations! You have performed a successful stake migration and your node will now continue to operate as normal. ## Troubleshooting In this section we cover several commonly seen issues encountered for staking nodes participating in the redistribution game. If you don't see your issue covered here or require additional guidance, check out the `#node-operators` [Discord channel](https://discord.com/channels/799027393297514537/811553590170353685) where you will find support from other node operators and community members. ### Frozen node A node will be frozen when the reserve commitment hash it submits in its [`commit` transaction](./../../concepts/incentives/redistribution-game.md) does not match the correct hash. The reserve commitment hash is used as proof that a node is storing the chunks it is responsible for. It will not be able to play in the redistribution game during the freezing period. See the [penalties](./../../concepts/incentives/redistribution-game.md) section for more information. #### Check frozen status You can check your node's frozen status using the `/redistributionstate` endpoint: ```bash curl -X GET http://localhost:1633/redistributionstate | jq ``` ```bash { "minimumFunds": "18750000000000000", "hasSufficientFunds": true, "isFrozen": false, "isFullySynced": true, "phase": "commit", "round": 176319, "lastWonRound": 176024, "lastPlayedRound": 176182, "lastFrozenRound": 0, "block": 26800488, "reward": "10479124611072000", "fees": "30166618102500000" } ``` The relevant fields here are `isFrozen` and `lastFrozenRound`, which respectively indicate whether the node is currently frozen and the last round in which the node was frozen. #### Diagnosing freezing issues In order to diagnose the cause of freezing issues we must compare our own node's status to that of other nodes within the same neighborhood by comparing the results from our own node returned from the `/status` endpoint to the other nodes in the same neighborhood which can be found from the `/status/peers` endpoint. First we check our own node's status: ```bash curl -s localhost:1633/status | jq ``` ```bash { "peer": "da7e5cc3ed9a46b6e7491d3bf738535d98112641380cbed2e9ddfe4cf4fc01c4", "proximity": 0, "beeMode": "full", "reserveSize": 3747532, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 183, "neighborhoodSize": 12, "batchCommitment": 133828050944, "isReachable": true } ``` And next we will find the status for all the other nodes in the same neighborhood as our own. ```bash curl -s localhost:1633/status/peers | jq ``` The `/status/peers` endpoint returns all the peers of our node, but we are only concerned with peers in the same neighborhood as our own node. Nodes whose `proximity` value is equal to or greater than our own node's `storageRadius` value all fall into the same neighborhood as our node, so the rest have been omitted in the example output below: ```bash { ... { "peer": "da33f7a504a74094242d3e542475b49847d1d0f375e0c86bac1c9d7f0937acc0", "proximity": 9, "beeMode": "full", "reserveSize": 3782924, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 188, "neighborhoodSize": 11, "batchCommitment": 133828050944, "isReachable": true }, { "peer": "da4b529cc1aedc62e31849cf7f8ab8c1866d9d86038b857d6cf2f590604387fe", "proximity": 10, "beeMode": "full", "reserveSize": 3719593, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 176, "neighborhoodSize": 11, "batchCommitment": 133828050944, "isReachable": true }, { "peer": "da5d39a5508fadf66c8665d5e51617f0e9e5fd501e429c38471b861f104c1504", "proximity": 10, "beeMode": "full", "reserveSize": 3777241, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 198, "neighborhoodSize": 12, "batchCommitment": 133828050944, "isReachable": true }, { "peer": "da4cb0d125bba638def55c0061b00d7c01ed4033fa193d6e53a67183c5488d73", "proximity": 10, "beeMode": "full", "reserveSize": 3849125, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 181, "neighborhoodSize": 13, "batchCommitment": 133828050944, "isReachable": true }, { "peer": "da4b1cd5d15e061fdd474003b5602ab1cff939b4b9e30d60f8ff693141ede810", "proximity": 10, "beeMode": "full", "reserveSize": 3778452, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 183, "neighborhoodSize": 12, "batchCommitment": 133827002368, "isReachable": true }, { "peer": "da49e6c6174e3410edad2e0f05d704bbc33e9996bc0ead310d55372677316593", "proximity": 10, "beeMode": "full", "reserveSize": 3779560, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 185, "neighborhoodSize": 12, "batchCommitment": 133828050944, "isReachable": true }, { "peer": "da4cdab480f323d5791d3ab8d22d99147f110841e44a8991a169f0ab1f47d8e5", "proximity": 10, "beeMode": "full", "reserveSize": 3778518, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 189, "neighborhoodSize": 11, "batchCommitment": 133828050944, "isReachable": true }, { "peer": "da4ccec79bc34b502c802415b0008c4cee161faf3cee0f572bb019b117c89b2f", "proximity": 10, "beeMode": "full", "reserveSize": 3779003, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 179, "neighborhoodSize": 10, "batchCommitment": 133828050944, "isReachable": true }, { "peer": "da69d412b79358f84b7928d2f6b7ccdaf165a21313608e16edd317a5355ba250", "proximity": 11, "beeMode": "full", "reserveSize": 3712586, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 189, "neighborhoodSize": 12, "batchCommitment": 133827002368, "isReachable": true }, { "peer": "da61967b1bd614a69e5e83f73cc98a63a70ebe20454ca9aafea6b57493e00a34", "proximity": 11, "beeMode": "full", "reserveSize": 3780190, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 182, "neighborhoodSize": 13, "batchCommitment": 133828050944, "isReachable": true }, { "peer": "da7b6a268637cfd6799a9923129347fc3d564496ea79aea119e89c09c5d9efed", "proximity": 13, "beeMode": "full", "reserveSize": 3721494, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 188, "neighborhoodSize": 14, "batchCommitment": 133828050944, "isReachable": true }, { "peer": "da7a974149543df1b459831286b42b302f22393a20e9b3dd9a7bb5a7aa5af263", "proximity": 13, "beeMode": "full", "reserveSize": 3852986, "pullsyncRate": 0, "storageRadius": 10, "connectedPeers": 186, "neighborhoodSize": 12, "batchCommitment": 133828050944, "isReachable": true } ] } ``` Now that we have the status for our own node and all its neighborhood peers we can begin to diagnose the issue through a series of checks outlined below: :::info If you are able to identify and fix a problem with your node from the checklist below, it's possible that your node's reserve has become corrupted. Therefore, after fixing the problem, stop your node, and repair your node according to the instructions in the section following the checklist. ::: 1. Compare `reserveSize` with peers The `reserveSize` value is the number of chunks stored by a node in its reserve. The value for `reserveSize` for a healthy node should be around +/- 1% the size of most other nodes in the neighborhood. In our example, for our node's `reserveSize` of 3747532, it falls within that normal range. This does not guarantee our node has no missing or corrupted chunks, but it does indicate that it is generally storing the same chunks as its neighbors. If it falls outside this range, see the next section for instructions on repairing reserves. 2. Compare `batchCommitment` with peers The `batchCommitment` value shows how many chunks would be stored if all postage batches were fully utilised. It also represents whether the node has fully synced postage batch data from on-chain. If your node's `batchCommitment` value falls below that of its peers in the same neighborhood, it could indicate an issue with your blockchain RPC endpoint that is preventing it from properly syncing on-chain data. If you are running your own node, check your setup to make sure it is functioning properly, or check with your provider if you are using a 3rd party service for your RPC endpoint. 3. Check `pullsyncRate` The `pullsyncRate` value measures the speed at which a node is syncing chunks from its peers. Once a node is fully synced, `pullsyncRate` should go to zero. If `pullsyncRate` is above zero it indicates that your node is still syncing chunks, so you should wait until it goes to zero before doing any other checks. If `pullsyncRate` is at zero but your node's `reserveSize` does not match its peers, you should check whether your network connection and RPC endpoint are stable and functioning properly. A node should be fully synced after several hours at most. 4. Check most recent `block` number The `block` value returned from the `/redistributionstate` endpoint shows the most recent block a node has synced. If this number is far behind the actual more recent block then it indicates an issue with your RPC endpoint or network. If you are running your own node, check your setup to make sure it is functioning properly, or check with your provider if you are using a 3rd party service for your RPC endpoint. ```bash curl -X GET http://localhost:1633/redistributionstate | jq ``` ```bash { "minimumFunds": "18750000000000000", "hasSufficientFunds": true, "isFrozen": false, "isFullySynced": true, "phase": "commit", "round": 176319, "lastWonRound": 176024, "lastPlayedRound": 176182, "lastFrozenRound": 0, "block": 26800488, "reward": "10479124611072000", "fees": "30166618102500000" } ``` 5. Check peer connectivity Compare the value of your node's `neighborhoodSize` from the `/status` endpoint and the `neighborhoodSize` of its peers in the same neighborhood from the `/status/peers` endpoint. The figure should be generally the same (although it may fluctuate slightly up or down at any one point in time). If your node's `neighborhoodSize` value is significantly different and remains so over time then your node likely has a connectivity problem. Make sure to [check your network environment](./../installation/connectivity.md) to ensure your node is able to communicate with the network. If no problems are identified during these checks it likely indicates that your node was frozen in error and there are no additional steps you need to take. ### Repairing corrupt reserve If you have identified and fixed a problem causing your node to become frozen or have other reason to believe that your node's reserves are corrupted then you should repair your node's reserve using the `db repair-reserve` command. First stop your node, and then run the following command: :::caution Make sure to replace `/home/bee/.bee` with your node’s data directory if it differs from the one shown in the example. Make sure that the directory you specify is the root directory for your node’s data files, not the localstore directory itself. This is the same directory specified using the `data-dir` option in your node’s [configuration](./configuration.md). ::: ```bash bee db repair-reserve --data-dir=/home/bee/.bee ``` After the command has finished running, you may restart your node. ### Node occupies unusually large space on disk During normal operation of a Bee node, it should not take up more than ~30 GB of disk space. In the rare cases when the node's occupied disk space grows larger, you may need to use the compaction `db compact` command. :::danger To prevent any data loss, operators should run the compaction on a copy of the localstore directory and, if successful, replace the original localstore with the compacted copy. ::: The command is available as a sub-command under db as such (make sure to replace the value for `--data-dir` with the correct path to your bee node's data folder if it differs from the path shown in the example): ```bash bee db compact --data-dir=/home/bee/.bee ``` ### Node not participating in redistribution First check that the node is fully synced, is not frozen, and has sufficient funds to participate in staking. To check node sync status, call the `redistributionstate` endpoint: ``` curl -X GET http://localhost:1633/redistributionstate | jq ``` Response: ```bash { "minimumFunds": "18750000000000000", "hasSufficientFunds": true, "isFrozen": false, "isFullySynced": true, "phase": "commit", "round": 176319, "lastWonRound": 176024, "lastPlayedRound": 176182, "lastFrozenRound": 0, "block": 26800488, "reward": "10479124611072000", "fees": "30166618102500000" } ``` Confirm that `hasSufficientFunds` is `true`, and `isFullySynced` is `true` before moving to the next step. If `hasSufficientFunds` is `false`, make sure to add at least the amount of xDAI shown in `minimumFunds` (unit of 1e-18 xDAI). If the node was recently installed and `isFullySynced` is `false`, wait for the node to fully sync before continuing. After confirming the node's status, continue to the next step. #### Run sampler process to benchmark performance One of the most common issues affecting staking is the `sampler` process failing. The sampler is a CPU-intensive process which is run by nodes which are selected to take part in redistribution. It does not need much memory, but on a slow processor or a slow disk it may fail or time out; 4 cores are sufficient. To check a node's performance, run `swarm-cli utility rchash`, or call the `/rchash` endpoint of the API directly. See the `/rchash` section of the [Bee API page for usage details](./bee-api.md#rchash). If you are still experiencing problems, you can find more help in the [node-operators](https://discord.gg/kHRyMNpw7t) Discord channel (for your safety, do not accept advice from anyone sending a private message on Discord). --- ## Swarm CLI **Swarm‑CLI** is a command‑line tool powered by `bee-js` that makes it easy to interact with your Bee node directly from the command line. It’s friendlier than working with the raw Bee HTTP API and faster than writing a custom `bee-js` script when you just want to perform an action from the terminal. :::tip `swarm-cli` is the recommended method for interaction with your Bee node from the command line. Unless you have explicit need to use the Bee API directly, `swarm-cli` is generally the better option. ::: Common uses: * Check your node: `swarm-cli status` * Add stake: * Upload files or a static site: `swarm-cli upload ` (will prompt to pick or create a postage batch) * Download content: `swarm-cli download -o ` * Inspect and manage postage batches: `swarm-cli ...` (use `--help` to see stamp-related commands) **Why use it?** * **No scaffolding needed** — run direct commands without creating a project * **Interactive prompts** — it guides you through common tasks such as stamp purchasing and selection using interactive prompts * **Smart option inference** — it infers options based on your input (e.g., batch selection, index page, content type) so you don’t need deep Bee API knowledge * **Powered by `bee-js`** — stays aligned with the latest Bee features It also greatly simplifies certain more complex tasks, such as as the management of [feeds](./../../develop/tools-and-features/feeds.md). For installation and usage instructions, [see the README](https://github.com/ethersphere/swarm-cli/blob/master/README.md). To check the latest version, see the Swarm CLI [releases page](https://github.com/ethersphere/swarm-cli/releases). For further support and information, [join the Swarm Discord server](https://discord.com/invite/GU22h2utj6). --- ## Uninstalling Bee Choose the appropriate uninstallation method based on how Bee was installed: ## Package Manager This method can be used for package manager based [installs](./../installation/package-manager.md) of the official Debian, RPM, and Homebrew packages. :::danger Uninstalling Bee will permanently delete your keyfiles and configuration. Ensure you have a [full backup](./backups.md) before proceeding. ::: ### Debian To uninstall Bee and completely remove all associated files including keys and configuration, run: ```bash sudo apt-get purge bee ``` ### RPM ```bash sudo yum remove bee ``` ## Shell Script / Binary Install If Bee was installed using the [automated shell script](./../installation/shell-script.md) or as a binary by [building from source](./../installation/build-from-source.md), it can be uninstalled by manually removing the installed binary, configuration files, and data directories. ### Identify Data and Config Locations The shell script install method may result in slightly different default data and configuration locations based on your system. The easiest way to find these locations is to check the default configuration using the `bee printconfig` command: ```bash bee printconfig ``` The output from this command contains several dozen default configuration values, however we only include the two we need in the example output below, `config` and `data-dir`. These will reveal the default locations for the configuration files and data directory according to our specific system. Your output should look similar to this: ```bash # config file (default is $HOME/.bee.yaml) config: /home/noah/.bee.yaml # data directory data-dir: /home/noah/.bee ``` ## Remove Configuration Files Bee does not automatically generate a configuration file, but it looks for one at **`$HOME/.bee.yaml`** by default. ### Check for Configuration Files **Default location for shell script installs:** ```bash ls -l $HOME/.bee.yaml ``` ### Remove Configuration Files If the files exist, remove them: ```bash rm -f $HOME/.bee.yaml ``` ### Verify Removal Run the following commands to ensure the configuration files have been deleted: ```bash ls -l $HOME/.bee.yaml ``` If the command returns **"No such file or directory"**, the configuration file has been successfully removed. :::caution If you have generated a config file and saved it to a non default location which you specify when starting your node using a command line flag (`--config`) or environment variable (`BEE_CONFIG`), then it is up to you to keep track of where you saved it and remove it yourself. ::: ## Remove Data Files Bee stores its **node data, blockchain state, and other persistent files** in a data directory. If you want to fully remove Bee, this directory must be deleted. The data directory [default location](./configuration.md#default-data-and-config-directories) differs based on install method and system type. ## Verify Uninstallation To confirm that Bee has been fully uninstalled, run: ```bash command -v bee ``` If Bee is still installed, this command will return the binary path (e.g., /usr/bin/bee). If it returns nothing, Bee has been successfully uninstalled. --- ## Upgrading Bee It's very important to keep Bee up to date to benefit from security updates and ensure you are able to properly interact with the Swarm network. The [#node-operators](https://discord.com/channels/799027393297514537/811553590170353685) channel is an excellent resource for any of your questions regarding node operation. :::warning Bee sure to [back up](./backups.md) your keys and [cash out your cheques](./cashing-out.md) to ensure your xBZZ is safe before applying updates. ::: :::warning Nodes should not be shut down or updated in the middle of a round they are playing in as it may cause them to lose out on winnings or become frozen. To see if your node is playing the current round, check if `lastPlayedRound` equals `round` in the output from the [`/redistributionstate` endpoint](/api/#tag/RedistributionState/paths/~1redistributionstate/get). See [staking section](./staking.md) for more information on staking and troubleshooting. ::: ## Version compatibility and upgrade path The Swarm network has a **minimum supported Bee version**. The minimum supported version is currently **v2.8.0**, the release that introduced a breaking p2p protocol change, so nodes running an older protocol can no longer connect to the network. When upgrading across a breaking protocol change, do not skip the release that introduced it. Upgrade *through* that version so any one-time data migrations run while they still exist in the code, since Bee removes old migration and compatibility code once a version is no longer supported. Bee v2.8.1 is **non-disruptive for nodes already on v2.8.0**: it makes no breaking p2p protocol changes, so you can upgrade in place using the steps below. :::warning **If you are running Bee v2.6.0 or older:** Bee v2.8.1 removes the last of the v2.6.0 backward-compatibility code, so you cannot upgrade to it directly. Either upgrade stepwise (**v2.6.0 → v2.8.0 → v2.8.1**) so the data migrations run, or reinstall the node fresh on v2.8.1. ::: ### Ubuntu / Debian To upgrade Bee, first stop the Bee service: ```bash sudo systemctl stop bee ``` Next, upgrade the `bee` package: ```bash sudo apt-get update sudo apt-get upgrade bee ``` And will see output like this after a successful upgrade: ``` Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following packages will be upgraded: bee 1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Need to get 0 B/27.2 MB of archives. After this operation, 73.7 kB of additional disk space will be used. Do you want to continue? [Y/n] Y (Reading database ... 103686 files and directories currently installed.) Preparing to unpack .../archives/bee_2.0.0_amd64.deb ... Unpacking bee (2.0.0) over (1.17.3) ... Setting up bee (2.0.0) ... Installing new version of config file /etc/default/bee ... ``` Make sure to pay attention to any prompts, read them carefully, and respond to them with your preference. You may now start your node again: ```bash sudo systemctl start bee ``` ### Manual Installations To upgrade your manual installation, simply stop Bee, replace the Bee binary and restart. ### Docker To upgrade your Docker installation, simply increment the version number in your configuration and restart. --- ## Bee FAQ ## Running a Bee Node ### How can I become part of the Swarm network? You can become part of the network by running a bee node. Bee is a peer-to-peer client that connects you with other peers all over the world to become part of the Swarm network, a global distributed p2p storage network that aims to store and distribute all of the world's data Depending on your needs you can run an ultra-light, light or full node. ### What are the differences between Bee node types? A bee node can be configured to run in various modes based on specific use cases and requirements. [See here](./working-with-bee/node-types.md#node-types-overview) for an overview of the differences. ### What are the requirements for running a Bee node? See the [getting started section](./installation/getting-started.md#requirements) for more information about running a Bee node. #### Full node All three node types run on ordinary consumer hardware. Full nodes use more disk space and bandwidth than the lighter modes and additionally need a Gnosis Chain connection and funds — see [full node specifications](./working-with-bee/node-types.md#full-node-specifications) for the current list. ### How much bandwidth is required for each node? Typically, each node requires around 10 megabits per second (Mbps) of bandwidth during normal operation. ### How do I Install Bee on Windows? Bee is compatible with Windows and a Bee `.exe` file can be found on the [`releases` page](https://github.com/ethersphere/bee/releases) of the Bee repo. It is also possible to [build from the source](./installation/build-from-source.md). ### How do I get the node's wallet's private key (use-case for Desktop app)? See the [backup section](./working-with-bee/backups.md) for more info. ### How do I import my private key to Metamask? You can import the `swarm.key` json file in MetaMask using your password file or the password you have set in your bee config file. ### Where can I find my password? You can find the password in the root of your data directory. See the [backup section](./working-with-bee/backups.md) for more info. ## Connectivity ### Which p2p port does Bee use and which should I open in my router? The default p2p port for Bee is 1634, please forward this using your router and allow traffic over your firewall as necessary. Bee also supports UPnP but it is recommended you do not use this protocol as it lacks security. For more detailed information see the connectivity section in the docs. https://docs.ethswarm.org/docs/bee/installation/connectivity ### How do I know if I am connected to other peers? You may communicate with your Bee using its HTTP api. Type `curl http://localhost:1633/peers` at your command line to see a list of your peers. ## Errors ### What does "could not connect to peer" mean? The "Could not connect to peer" error can occur for various reasons. One of the most common is that you have the identifier of a peer in your address book from a previous session. When trying to connect to this node again, the peer may no longer be online. ### What does "context deadline exceeded" error mean? The "context deadline exceeded" is a non-critical warning. It means that a node took unexpectedly long to respond to a request from your node. Your node will automatically try again via another node. ### How do I set up a blockchain endpoint? We recommend you run your own [Gnosis Node using Nethermind](https://docs.gnosischain.com/node/tools/sedge). - If you use "bee start" - you can set it in your bee configuration under --blockchain-rpc-endpoint or BEE_BLOCKCHAIN_RPC_ENDPOINT - open ~/.bee.yaml - set `blockchain-rpc-endpoint: http://localhost:8545` - If you use bee.service - you can set it in your bee configuration under --blockchain-rpc-endpoint or BEE_BLOCKCHAIN_RPC_ENDPOINT - open /etc/bee/bee.yaml - and then uncomment `blockchain-rpc-endpoint` configuration - and set it to `http://localhost:8545` - after that sudo systemctl restart bee ### How can I export my private keys? See the section on [backups](./working-with-bee/backups.md) for exporting your keys. ### How to import bee node address to MetaMask? 1. See the [backup section](./working-with-bee/backups.md) for info on exporting keys. 2. Go to Metamask and click "Account 1" --> "Import Account" 3. Choose the "Select Type" dropdown menu and choose "JSON file" 4. Paste the password (Make sure to do this first) 5. Upload exported JSON file 6. Click "Import" ### What are the restart commands of bee? If you use bee.service: - Start: `sudo systemctl start bee.service` - Stop: `sudo systemctl stop bee.service` - Status: `sudo systemctl status bee.service` If you use "bee start" - Start: `bee start` - Stop: `ctrl + c` or `cmd + c` or close terminal to stop process ### Relevant endpoints and explanations See the [API Reference](https://docs.ethswarm.org/api/) pages for details. ### How can I check how many cashed out cheques do I have? You can look at your chequebook contract at etherscan. Get your chequebook contract address with: `curl http://localhost:1633/chequebook/address` ### Where can I find documents about the cashout commands? Learn how to cash out [here](./working-with-bee/cashing-out.md). ### When I run http://localhost:1633/chequebook/balance I get "totalBalance" and "availableBalance" what is the difference? `totalBalance` is the balance on the blockchain, and `availableBalance` is that balance minus the outstanding (non-cashed) cheques that you have issued to your peers. These latter cheques do not show up on the blockchain. It's like what the bank thinks your balance is vs what your chequebook knows is actually available because of the cheques you've written that are still "in the mail" and not yet cashed. ### What determines the number of peers and how to influence their number? Why are there sometimes 300+ peers and sometimes 30? The number of connected peers is determined by your node as it attempts to keep the distributed Kademlia well connected. As nodes come and go in the network your peer count will go up and down. If you watch bee's output logs for "successfully connected", there should be a mix of (inbound) and (outbound) at the end of those messages. If you only get (outbound) then you may need to get your p2p port opened through your firewall and/or forwarded by your router. Check out the connectivity section in the docs https://docs.ethswarm.org/docs/bee/installation/connectivity. ### What is the difference between "systemctl" and "bee start"? _bee start_ and _systemctl start bee_ actually run 2 different instances with 2 different _bee.yaml_ files and two different data directories. _bee start_ uses _~/.bee.yaml_ and the _~/.bee_ directory for data _systemctl_ uses _/etc/bee/bee.yaml_ and (IIRC) _/var/lib/bee_ for data ## Swarm Protocol ### Can I use one Ethereum Address/Wallet for many nodes? No, this violates the requirements of the Swarm Protocol and will break critical node functions such as staking, purchasing stamp batches, and uploading data. Therefore, the rule is, each node must have: - 1 Ethereum address (this address, the Swarm network id, and a random nonce are used to determine the node's overlay address) - 1 Chequebook - 2 Unique ports for Bee API / p2p API ## Miscellaneous ### How can I add Gnosis / Sepolia to Metamask? You can easily add Sepolia or Gnosis to metamask using the [official guide from Metamask](https://support.metamask.io/configure/networks/how-to-add-a-custom-network-rpc/). If you are using a different wallet which does not have an easy option for adding networks like Metamask does, then you may need to add the networks manually. You need to fill in four pieces of information to do so: #### Gnosis Chain Network name: Gnosis RPC URL: https://xdai.fairdatasociety.org Chain ID: 100 Currency symbol: XDAI --- ## Add Access Control :::info This is guide contains a detailed explanation of how to use the ACT feature, but does not cover its higher level concepts. To better understand how ACT works and why to use it, read [the ACT page in the "Concepts" section](./../concepts/access-control.md). ::: In this section we'll provide information on how to use the **swarm-cli** to upload, download data with ACT or update the grantee list. ## Upload Uploading data without ACT to the network remains unchanged. To upload with ACT use the **act** and **act-history-address** flags following the **upload** command: ```bash swarm-cli upload test.txt --act --stamp $stamp_id --act-history-address $swarm_history_address ``` Here **act** indicates that the file provided shall be uploaded using ACT. The **act-history-address** flag is the reference of the historical version of the ACT. It can be omitted, in which case the data is uploaded to a new history. If provided, then the data will be uploaded to that history as the latest version. In both cases the timestamp of the upload is taken as the key of the history entry. If the provided **act-history-address** is invalid then the request will fail with a not found error. The response returns the newly created reference encrypted with ACT and the header contains history reference. ## Download Downloading data which was uploaded without ACT from the network remains unchanged. To download with ACT use the **act**, **act-publisher**, **act-timestamp** and **act-history-address** flags following the **download** command: ```bash swarm-cli download $swarm_hash test.txt --act --act-history-address $swarm_history_address --act-publisher $public_key --timestamp $timestamp ``` Here **act** indicates that the **swarm_hash** shall be decrypted using the content publisher's public key as **act-publisher** and the lookup table mentioned above. The **act-history-address** flag is the reference of the historical version of the ACT based on the timestamp provided, however the **act-timestamp** flag can be omitted in which case the current timestamp is used. If the **act-history-address** or **act-publisher** flags are omitted then the request is treated as a "usual" download. If the data was uploaded with ACT and we try to download it without the ACT flags then the request will fail with a not found error. ## Grantee management Updating a grantee list literally means patching a json file containing the list of grantee swarm public keys. ### Create A brand new grantee list can be created using the following command: ```bash swarm-cli grantee create grantees.json --stamp $stamp_id ``` where **grantees.json** shall contain the key **grantees** with the list of public keys: ```json { "grantees": [ "03ec55e9fb2aefb8600f69142abaad79311516c232b28919d66efb4d41bce15bfa", "03fdcab22b455ce08a481d929a4cb9f447752545818eded1ad1785c51581e822c6" ] } ``` The response returns the newly created and encrypted grantee list and the history reference. Only the publisher can decrypt and therefore access the list. If **act-history-address** is provided then the grantee list is uploaded as the newest version under that history. ### Patch ```bash swarm-cli grantee patch grantees-patch.json --reference $grantee_reference --history $grantee_history_reference --stamp $stamp_id ``` where **grantees.json** shall contain the keys **add** and **revoke** with the list of public keys for granting and revoking access, respectively: ```json { "add": ["03fdcab22b455ce08a481d929a4cb9f447752545818eded1ad1785c51581e822c6"], "revoke": [ "03ec55e9fb2aefb8600f69142abaad79311516c232b28919d66efb4d41bce15bfa" ] } ``` The **reference** flag indicates the already existing encrypted grantee list reference that needs to be updated. The **grantee_history_reference** indicates the reference of historical version of the list, where the encrypted list reference is added as a metadata to the history entry with the key **"encryptedglref"** **Limitation**: If an update is called again within a second from the latest upload/update of a grantee list, then mantaray save fails with an invalid input error, because the key (timestamp) already exists, hence a new fork is not created. ### Get As stated above, only the publisher can decrypt and therefore access the list with the following command: ```bash swarm-cli grantee get $grantee_reference ``` which simply returns the latest version of the list. Non-authorized access causes the request to fail with a not found error. For each of the above operations, if the provided **act-history-address** or **reference** is invalid then the request will fail with a not found error. --- ## Contribute to Bee Development Bee is developed in the open on GitHub, and contributions are welcome via pull request. We love PRs! 🐝 We would love you to get involved with our [Github repo](https://github.com/ethersphere/bee). Connect with other Bee developers over at the official [Discord Server](https://discord.gg/kHRyMNpw7t). Sign up and get involved with our buzzing hive of daily dev chat. - If you would like to contribute, please read the [coding guidelines](https://github.com/ethersphere/bee/blob/master/CODING.md) before you get started. - Installation from source is described in the [Installation](./../../bee/installation/build-from-source.md). - Contribute to Swarm’s evolution by proposing your own Swarm Improvement Proposal (SWIP) [here](https://github.com/ethersphere/SWIPs). ## Testing a connection with PingPong protocol To check if two nodes are connected and to see the round trip time for message exchange between them, get the overlay address from one node, for example local node 2: ```bash curl localhost:1833/addresses ``` Make sure addresses are configured as in examples above. And use that address in the API call on another node, for example, local node 1: ```bash curl -X POST localhost:1735/pingpong/d4440baf2d79e481c3c6fd93a2014d2e6fe0386418829439f26d13a8253d04f1 ``` ## Generating protobuf To process protocol buffer files and generate the Go code from it two tools are needed: - [protoc](https://github.com/protocolbuffers/protobuf/releases) - [protoc-gen-gogofaster](https://github.com/gogo/protobuf) Makefile rule `protobuf` can be used to automate `protoc-gen-gogofaster` installation and code generation: ```bash make protobuf ``` --- ## Protocols ## Protocols specifications An attempt to describe the desired behaviour of the main DISC protocols, which corresponds to the rational choice a node should make in order to maximize its profitability. A communications protocol is a set of formal rules describing how to transmit or exchange data, especially across a network. We'll begin by describing the Hive, Retrieval, PushSync, PullSync communication protocols as well as the Kademlia topology component. ## Proposal The purpose of this document is to specify these concepts: - A pattern of exchange of messages which in semantic units corresponds to the high level function of what a node accomplishes in an exchange. - Strategies of behaviour that a node should adopt in situations like network disconnects, timeouts, invalid chunks etc. - An incentivisation strategy such that constructive behaviour should be rewarded and encouraged while deviating from the protocol rules should result in punishing measures. ## Hive The Hive protocol defines how nodes exchange information about their peers in order to reach and maintain a saturated Kademlia connectivity. The exchange of this information happens upon connection, however nodes can broadcast newly received peers to their peers during the lifetime of connection. While the simplest approach is to share all known peers (during an exchange) it might be more optimal to narrow down to a useful subset of peers - for instance all the peers up to a certain depth or belonging to a certain bin. The exchanged information includes both overlay and underlay addresses of the known remote peers. The overlay address serves to select peers to achieve the connectivity pattern needed for the desired network topology, while the underlay address is needed to establish the peer connections by dialing selected peers. Upon receiving a peers message, nodes should store the peer information in their address book, i.e., a data structure containing info about peers known to the node that is meant to be persisted across sessions. ### Appendix The protobuf definitions ```protobuf // Copyright 2020 The Swarm Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. syntax = "proto3"; package hive; option go_package = "pb"; message Peers { repeated BzzAddress peers = 1; } message BzzAddress { bytes Underlay = 1; bytes Signature = 2; bytes Overlay = 3; bytes Nonce = 4; } ``` ## Kademlia Kademlia topology is a specific connectivity pattern used by all DISC protocols and its purpose is to route messages between nodes in a network using overlay addressing. The message routing happens in such a fashion that with every network hop we will get closer to the target node, specifically at half of the distance covered by the previous hop. Swarm uses the recursive/forwarding style of Kademlia. This approach implies that every forwarding node - once it received a request - will keep an in-memory record that captures the request related information (requester, time of the request etc.) until the request is satisfied, rejected or times out. Because a forwarder can not reliably tell how much time the downstream peer will need to satisfy the request - the choice of a reasonable value for waiting period is a point of contention. The choice of a reasonable waiting period is constrained by these factors: - keeping the in-memory record for too long means that there's going to be a limit on how many concurrent requests a peer can keep "in-flight", because memory is limited. - if the peer decides to time out prematurely (while downstream peers are still processing the request) then the effort of all the downstream peers will be wasted. - we should distinguish between unsolicited chunks and chunks that we received from the downstream after we stopped waiting for the response (timed out). After a certain period of time all responses will be treated the same because of the need to free the allocated resources (the in-memory record). Conversely the downstream should be informed when the upstream is no longer interested in the previously sent request, so it could free the used resources. This way the downstream won't return the chunk after the request has timed out, risking being punished. ## Push and pull: chunk retrieval and syncing Swarm involves a direct storage scheme of fixed size where chunks are stored on nodes with address corresponding to the chunk address. The syncing protocols act in such a way that they reach those neighborhoods whenever a request is initiated. Such a route is sure to exist as a result of the Kademlia topology of keep-alive connections between peers. The process of relaying the request from the initiator to the storer is called forwarding and also the process of passing the chunk data along the same path is called backwarding. Conversely - Backwarding and Forwarding are both notions defined on a keep alive network of peers as strategies of reaching certain addresses. If we zoom into a particular node in the forwarding (or backwarding) path we see the following strategy: - Receive a request - Decide who to forward the request to (decision strategy) - Have a way to match the the response to the original request. The crucial step is the second one - strategy of choosing the peer to forward the request to and how they react to failure like stream closure or nodes dropping offline or closing the protocol connection and whether we proactively initiate several requests to peers. The last step does not apply for the storer nodes, since they do not forward the request but they satisfy it. The key element of these notions is that the decision about the next action is being done on the node level, which will select the next peers(s) and delegate them with handling the request. The simplest representation of this would be a recursive algorithm that with every iteration gets closer to the target address and stops when it runs out of peers or successfully reaches the target node. ### Requirements - We need a way to determine the "best" candidate peer to forward the request to, and if this option fails, continue with the "next-best" candidate until we exhaust available peers. The decision of picking the "best" peer is delegate to an overlay driver that has the best knowledge of this peer's past history and performance and topology structure. - We need a strategy of parallelisation of requests that we pass downstream, where appropriate. Parallel requests to different peers allow us increase the chances of successfully syncing the chunk but it comes with the cost of using our bandwidth allowance, so it's imperative to zoom in on an optimal balance between the two. - We need a way to ensure that when we issue a syncing request we don’t end up in a situation when this request comes back around to us, wasting network resources. - In the case when we are a "forwarder" node, we might consider a decision strategy on whether we want to cache the chunk in the event of a repeated request. - To every forwarding/backwarding exchange we attach an incentivisation action that would take into account variables like the success of the action and the cost of performing the action. - We need to design the optimal incentivisation scheme, to determine the optimal payment/settlement frequency and correctness of computation of the payment/charged amount. This also applies to both chunk storage scheme and the relayed request-response scheme. - We need to have a sensible strategy when it comes to waiting for a peer to respond to our request; as a forwarder, we want to make our best effort to sync the chunk but without waiting for an excessive amount of time, which would lead to waste of resources. - When receiving a response to an expired request - or we are unable to conclude if such a request has ever been issued - punishing measures should be imposed on the upstream peer. ### Incentivisation strategy An incentivisation strategy should be put in place in such way that it encourages honest collaboration between nodes. This implies that a given peer will make the best effort to satisfy any request while not allowing any abuse and waste of its resources. Having an accounting component that would keep track of the exchange activity between peers ensures that we do not allow excessive freeloading from the misbehaving peers. Having a granular punishment strategy ensures that the peers who misbehave (perhaps due to network latencies) will not be sanctioned to the same extent as peers who engage in grave protocol breaches, but are given a chance to "clean up their act". ## Retrieval The retrieval of a chunk is a process which fetches a given chunk from the network by its address. Chunk retrieval follows the general semantics of chunk syncing and takes the same network path as the push sync protocol, but in reverse. ### Protocol breach - Receiving a repeated request for a non-existent chunk should lead to rate limiting in order to discourage resource wasteful actions. - Receiving a response in the form of an invalid chunk constitutes a protocol breach and punishing measures are being taken against the peer at fault. ```markdown step I 1) check if this exact request has been received within last N minutes and it is for a non-existent chunk 2) if such request is found - take punishing measures against the requester (blocklisting) 3) request a peer from Kademlia 4) request chunk from the peer 5) if the peer does not return a valid chunk - go back to step 3 6) if the chunk is found and valid, log the event details in the local state and return the chunk to the requester 7) consider caching the chunk in case there might be a repeated request for it Error states - if we exhaust the list of peers (candidates) for this action, return a 'failure to get chunk' response to the requester. We might consider increasing our peer connections pool to avoid such situation in the future - if we are able to conclude that the chunk is non-existent (TBD) we return 'chunk not found' and consider rate limiting measures against the requester. - if we ran out of allowed time while looking for the chunk we return a 'timeout' response to the requester - if the chunk is retrieved successfully but does not pass validation, take punishing measures against the peer (blocklisting). - if the attempt fails, log the relevant attempt details in the local state and repeat the attempt against a new peer - if the peer times out responding to our request we log the attempt details and repeat step II against a new peer ``` ### Request chunk - sequence diagram ```mermaid sequenceDiagram Originator->>+Backwarder: Request for chunk Backwarder->>+Backwarder: Have I seen this request before Backwarder->>-Originator: Reject duplicate request Backwarder->>+Backwarder: Request next peer from the Kademlia iterator Backwarder->>+Storer: Request for chunk Storer->>-Backwarder: Success Backwarder->>-Originator: Return chunk ``` ### Request chunk - flow diagram ```mermaid flowchart TD A[Get next peer from Kademlia] --> H{Any time left?} H --> |No| I(Reject request) --> S[STOP] H --> |Yes| M{Any peers left} M --> |Yes| N(Request the chunk from peer) --> K{Check response} M --> |No peers left to try| I K --> |Timeout| L(Update peer stats) --> A K --> |Invalid chunk| N1(Punish peer) --> A K --> |Success| R(Return the chunk to the upstream peer) --> S ``` ### Appendix The protobuf definitions ```protobuf // Copyright 2020 The Swarm Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. syntax = "proto3"; package retrieval; option go_package = "pb"; message Request { bytes Addr = 1; } message Delivery { bytes Data = 1; bytes Stamp = 2; } ``` ## Pushsync Pushsync protocol is responsible for ensuring delivery of the chunk to its prescribed storer after it has been uploaded to any arbitrary node. The Pushsync protocol works in a similar way to the Retrieval protocol: the chunk is passed to the peer whose address is closest to the chunk address, and a custody receipt is received in response. Then the same process is repeated until the chunk eventually reaches the storer node located in a certain "neighborhood". Since the Pushsync protocol is a "mirror" version of the Retrieval protocol - it ensures that a successfully uploaded chunk is retrievable from the same "neighborhood" by the virtue of the fact that nodes in a neighborhood are connected to each other. ### Multiplexing Multiplexing is a recommended node strategy for the push sync protocol that involves early replication and opportunistic receipting. Its intention is to reduce the dependence on single closest nodes and to improve on network performance, i.e., push sync success rate, bandwidth overhead and latency. #### Context The current implementation of the push sync protocol aims to push a chunk to the closest node in the neighborhood which is then supposed to give out a receipt. Pushing the chunk to the single closest node is motivated by the retrieval protocol, which aims to find the chunk at that closest node. When the closest node hands out a receipt, this node also replicates the chunk to 3 peers in the neighborhood which are further away from the chunk than him. This replication takes place to ensure that the chunk is not lost when the closest node shuts down before the chunk is not pull-sync'ed and to speed up the spreading of the chunk in the neighborhood, in advance of pull sync. #### Problem Treating the closest node as a single target of push sync is fragile. If this peer has a badly-performing blockchain backend, slow or incomplete connectivity or is malicious, it may not spread the chunk and/or does not respond with a receipt. In this case, currently, the originator must retry the entire push-sync operation many times before the other peers within neighborhood recognise the improper behaviour. In-neighborhood retries are ideally avoided because such retries might cause the downstream timeouts to expire. In case of incomplete connectivity, the push sync protocol can end at a different branch of the neighborhood than the retrieval protocol--causing the chunk not to be retrievable. It should be noted that the pull sync protocol (may?) remedies this problem with a small time-delay. #### Multiplexing: early replication within neighborhood The first node in the push sync forward chain that falls within the neighborhood acts as *multiplexer*, i.e., it forwards the request to a number of closest nodes and responds with a self-signed receipt. Thus in achieving retrievability and security via early replication, we do not critically rely on the closest node to be available any more. #### Push sync flow We define the different roles peers have as part of the push sync forwarding chain: - originator -- creator of the request - forwarder -- closer to the chunk than the originator, further away away than the 1-before node. - multiplexer -- first node in the forward chain who is in the neighborhood - closest nodes -- according to the downstream node (usually the multiplexer), within the `n` closest nodes to the chunks (not including self) we describe the envisioned flow of push sync by describing the intended behaviour strategy of the various roles. 1. originator sends chunk to a peer closer to the chunk. 2. forwarder(s) forwards chunk that ends up with a node already within the neighborhood that acts as multiplexer 3. The multiplexer concurrently sends the chunk to 3 closest nodes attaching a multiplexing-list as part of the protocol message. At the same time they respond to their upstream peer with a self-signed receipt (unless the multiplexer is itself the originator). 4. Non-multiplexing closest nodes, i.e., nodes in the neighborhood that receive the pushsync message from a not-closest neighbour with a multiplexing list included, validate whether, based on their view, the multiplexing list covers all 3 closest nodes (potentially including the peer and/or the upstream peer themselves). If not, the node forwards the chunk to the peers left out. These peers are also added to the multiplexing list received from upstream and the extended list is attached with the chunk pushed. If the multiplexer node does not know a closest peer *p* but several of its chosen closest nodes do, then that node *p* will receive the same pushsynced chunk multiple times ### Appendix The protobuf definitions ```protobuf // Copyright 2020 The Swarm Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. syntax = "proto3"; package pushsync; option go_package = "pb"; message Delivery { bytes Address = 1; bytes Data = 2; bytes Stamp = 3; } message Receipt { bytes Address = 1; bytes Signature = 2; bytes Nonce = 3; } ``` ## Pullsync While the other described protocols are request scoped, Pullsync is a subscription type protocol. It's worth mentioning that the chunks that are being synchronized between nodes always travel alongside their corresponding postage stamps. Pullsync's role is to help synchronization of the chunks between neighborhood nodes. It bootstraps new nodes by filling up their storage with the chunks in range of their storage radius and also ensures eventual consistency - by making sure that the chunks will gradually migrate to their storer nodes. There are two kinds of syncing: - historical syncing: catching up with content that arrived to relevant neighborhood before this session started (after an outage or for completely new nodes). - live syncing: fetching the chunks that are received after the session has started. The chunks are served in batches (ordered by timestamp) and they cover contiguous ranges. The downstream peers coordinate their syncing by requesting ranges from the upstream with the help of the "interval store" - to keep track of which ranges are left to be synchronized. Because live syncing happens in sessions - it is inevitable that after a session is completed - the downstream peer disconnects and will be missing chunks that arrive later. For this purpose the downstream peer will make a note about the timestamp of the last synced chunk on disconnect. The point of the interval based approach is to cover those gaps that inevitably arise in between syncing sessions. To save bandwidth, before the contents of the chunk is being sent over the wire, the upstream will sent a range of chunk addresses for approval. If the downstream decides that some (or all) addresses are desired - a confirmation message is sent to the upstream, to which it responds with the chunks mentioned in the request. ```mermaid sequenceDiagram Downstream->>+Upstream: Get Upstream-->>-Downstream: Offer address range Downstream->>+Upstream: Want address range Upstream-->>-Downstream: Delivery ``` ### Appendix The protobuf definitions ```protobuf // Copyright 2020 The Swarm Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. syntax = "proto3"; package pullsync; option go_package = "pb"; message Syn {} message Ack { repeated uint64 Cursors = 1; } message Get { int32 Bin = 1; uint64 Start = 2; } message Chunk { bytes Address = 1; bytes BatchID = 2; } message Offer { uint64 Topmost = 1; repeated Chunk Chunks = 2; } message Want { bytes BitVector = 1; } message Delivery { bytes Address = 1; bytes Data = 2; bytes Stamp = 3; } ``` ### Peer rating When choosing a peer in relation to a given address - in addition to the distance between them - the Kademlia component will take into account two other factors: - the historical performance of the given peer, both in terms of latencies and past occurrences of protocol misalignments. - the accounting aspect, peers with whom we have higher credit will be preferred. - we should also prioritise those downstream peers that managed to produce responses in a previously computed amount of time (that would take into consideration the average time needed for a hop multiplied by the expected number of hops needed to reach a target neighborhood). Kademila should be indexing peers by their proximity order and peers rating in order to prioritize peers based on their expected performance. ### Decision strategy An optimal decision strategy will take into account both proximity order and peer rating to select (out of all connected peers) the best one to pass down the request. At the implementation level the Kademlia component will offer (in exchange for a given address) a stateful iterator that the client (protocol) will use to get the "next-best" peer. ### Transport A reliable network transport is required for the proper functionality of DISC protocols. The network transport can be a distinct component responsible for ensuring delivery, retrying on network issues and timeouts, and making optimal use of network resources. One example of usage for such a component could be embedding into the Kademlia driver so that the topology component is only concerned with overlay related operations, abstracting away any low level transport concerns. --- ## Dynamic Content Every upload to Swarm produces a unique content hash — change one byte and you get a different address. This is great for data integrity, but it means there is no built-in way to give someone a single, stable link that always shows the latest version of your content. Feeds solve this problem. A feed acts as a mutable pointer on top of Swarm's immutable storage, giving you a permanent address that always resolves to whatever content you last pointed it at. If you followed the [Host a Webpage](/docs/develop/host-your-website) guide, you already used feeds to enable seamless website updates without changing your ENS content hash. This guide explains how feeds actually work under the hood and walks through building a simple dynamic application from scratch. ## Prerequisites * A running Bee node ([install guide](/docs/bee/installation/quick-start)) * A valid postage stamp batch ([how to get one](/docs/develop/tools-and-features/buy-a-stamp-batch)) * Node.js 18+ and `@ethersphere/bee-js` installed ## Example Scripts and Projects The `bee-js` code snippets throughout this guide are available as runnable scripts in the [examples](https://github.com/ethersphere/examples) repo. The guide also includes a complete blog project that puts all the concepts together. The full working scripts are available in the [examples](https://github.com/ethersphere/examples) repo: * [`script-01.js`](https://github.com/ethersphere/examples/blob/main/dynamic-content/script-01.js) — The Immutability Problem * [`script-02.js`](https://github.com/ethersphere/examples/blob/main/dynamic-content/script-02.js) — Write, Read, and Update a Feed * [`script-03.js`](https://github.com/ethersphere/examples/blob/main/dynamic-content/script-03.js) — Feed Manifests — Stable URLs The complete blog project is in the [`simple-blog`](https://github.com/ethersphere/examples/tree/main/simple-blog) directory. Clone the repo and set up the example scripts: ```bash git clone https://github.com/ethersphere/examples.git cd examples/dynamic-content npm install cp .env.example .env ``` Fill in your `BATCH_ID` and verify `BEE_URL` in `.env`: ```bash BEE_URL=http://localhost:1633 BATCH_ID= ``` You can then run any script with: ```bash node script-01.js node script-02.js node script-03.js ``` The blog project has its own setup — see [Example Project — Simple Blog](#example-project--simple-blog) below. ## The Immutability Problem To see why feeds are necessary, try uploading the same content twice with a small change ([`script-01.js`](https://github.com/ethersphere/examples/blob/main/dynamic-content/script-01.js)): ```js const bee = new Bee("http://localhost:1633"); const batchId = "BATCH_ID"; const upload1 = await bee.uploadFile(batchId, "Hello Swarm - version 1", "note.txt"); console.log("Version 1:", upload1.reference.toHex()); const upload2 = await bee.uploadFile(batchId, "Hello Swarm - version 2", "note.txt"); console.log("Version 2:", upload2.reference.toHex()); ``` Each upload returns a different hash. If you shared the first hash with someone, they would always see "version 1" — there is no way to redirect them to "version 2" using content addressing alone. Feeds provide the missing layer of indirection. ## Feeds — Mutable Pointers on Immutable Storage A feed is identified by two things: an **owner** (an Ethereum address derived from a private key) and a **topic** (a human-readable string that you choose, like `"my-website"` or `"notes"`). Together, these uniquely identify a feed on the network. The feed owner can write Swarm references to the feed sequentially — first at index 0, then index 1, and so on. Anyone who knows the owner and topic can read the feed and retrieve the latest reference. The feed itself does not store your content directly; it stores a *pointer* (a Swarm reference) to content that you uploaded separately. :::info Feeds are built on top of [single-owner chunks](/docs/develop/tools-and-features/chunk-types#single-owner-chunks), a special chunk type in Swarm where the address is derived from an identity rather than the content. For a deeper look at how this works, see the [bee-js SOC and Feeds documentation](https://bee-js.ethswarm.org/docs/soc-and-feeds/). ::: :::warning Always use immutable stamp batches with feeds When a [mutable batch](/docs/concepts/incentives/postage-stamps#mutable-batches) fills up, new chunks overwrite the oldest chunks in each bucket. If feed entry chunks get overwritten, the sequential indexing scheme that feeds depend on breaks — lookups will fail because earlier indices are no longer reachable. Always use an **immutable** batch when working with feeds. ::: ### Create a Publisher Key Before creating a feed, you need a dedicated private key that will sign feed updates. Anyone with this key can publish to your feed, so store it securely. ```js const hex = "0x" + crypto.randomBytes(32).toString("hex"); const pk = new PrivateKey(hex); console.log("Private key:", pk.toHex()); console.log("Address:", pk.publicKey().address().toHex()); ``` Save the private key somewhere secure. You will use it for all future feed updates. ### Write and Read a Feed Now upload some content and write its reference to a feed, then read it back ([`script-02.js`](https://github.com/ethersphere/examples/blob/main/dynamic-content/script-02.js)): ```js const bee = new Bee("http://localhost:1633"); const batchId = "BATCH_ID"; const pk = new PrivateKey("YOUR_PRIVATE_KEY"); const owner = pk.publicKey().address(); // Choose a topic for this feed const topic = Topic.fromString("notes"); // Upload content to Swarm const upload = await bee.uploadFile(batchId, "My first note", "note.txt"); console.log("Content hash:", upload.reference.toHex()); // Write the content reference to the feed const writer = bee.makeFeedWriter(topic, pk); await writer.upload(batchId, upload.reference); console.log("Feed updated at index 0"); // Brief pause to allow the node to index the feed chunk await new Promise((r) => setTimeout(r, 1000)); // Read the latest reference from the feed const reader = bee.makeFeedReader(topic, owner); const result = await reader.downloadReference(); console.log("Latest reference:", result.reference.toHex()); console.log("Current index:", result.feedIndex.toBigInt()); ``` ### Update the Feed When you have new content, upload it and write the new reference to the feed. The writer automatically uses the next sequential index (this continues from the previous snippet — both are combined in [`script-02.js`](https://github.com/ethersphere/examples/blob/main/dynamic-content/script-02.js)): ```js // Upload updated content const upload2 = await bee.uploadFile(batchId, "My updated note", "note.txt"); console.log("New content hash:", upload2.reference.toHex()); // Update the feed — writer auto-discovers the next index await writer.upload(batchId, upload2.reference); console.log("Feed updated at index 1"); // Brief pause to allow the node to index the new entry await new Promise((r) => setTimeout(r, 1000)); const result2 = await reader.downloadReference(); console.log("Latest reference:", result2.reference.toHex()); console.log("Current index:", result2.feedIndex.toBigInt()); // 1n ``` ## Feed Manifests — Stable URLs So far, reading a feed requires knowing the owner address and topic. A **feed manifest** packages these two values into a single Swarm hash that acts as a permanent URL. When Bee resolves a feed manifest through the `/bzz/` endpoint, it automatically looks up the latest feed entry and serves whatever content it points to ([`script-03.js`](https://github.com/ethersphere/examples/blob/main/dynamic-content/script-03.js)). ```js // Create a feed manifest (one-time operation) const manifest = await bee.createFeedManifest(batchId, topic, owner); console.log("Feed manifest:", manifest.toHex()); ``` You can now access the content through a stable URL: ```text http://localhost:1633/bzz/FEED_MANIFEST_HASH/ ``` Every time you update the feed, the same URL serves the new content — no URL change needed. This is also the hash you would register in ENS as your content hash (see [Host a Webpage - Connect to ENS](/docs/develop/host-your-website#optional-connect-site-to-ens-domain)). :::tip A feed manifest only needs to be created once. After that, just update the feed and the manifest URL will always resolve to the latest content. ::: ## How It All Fits Together The resolution chain when someone accesses your feed manifest URL: ```text GET /bzz/MANIFEST_HASH/ → Bee downloads the manifest, extracts the topic and owner → Looks up the latest feed entry for that topic/owner pair → Reads the Swarm content reference from the latest entry → Retrieves and serves the content at that reference ``` From the outside, a feed manifest URL behaves exactly like a regular Swarm URL — except the content behind it can change whenever the feed owner publishes an update. ## Example Project — Simple Blog This section puts everything together into a minimal but complete project: a **simple blog** that lives on Swarm. It generates a static HTML site with an index page listing all posts and individual post pages. The publisher can create, edit, and delete posts — and readers always find the latest version at a single stable URL. :::info This project follows the same architectural pattern used by [Etherjot](https://github.com/ethersphere/etherjot), a full-featured blogging platform on Swarm. Etherjot regenerates and re-uploads the entire blog site each time a post is added, then updates a single feed to point to the new version. Our blog does the same thing in a simplified form. ::: ### Project Setup Clone the [examples](https://github.com/ethersphere/examples) repo (if you haven't already) and navigate to the blog project: ```bash git clone https://github.com/ethersphere/examples.git cd examples/simple-blog npm install ``` Copy `.env.example` to `.env` and fill in your `BATCH_ID`: ```bash cp .env.example .env ``` ```bash BEE_URL=http://localhost:1633 BATCH_ID= ``` ### Project Structure ``` simple-blog/ ├── .env # Bee URL and batch ID ├── html.js # Shared HTML generation utility ├── init.js # Initialize the blog (run once) ├── post.js # Create, edit, or delete a post and update the feed ├── read.js # Read the feed (demonstrates reader access) ├── config.json # Generated by init.js — stores keys and manifest hash └── posts.json # Generated by init.js — stores all blog posts ``` ### The HTML Generation Module The blog regenerates its site from `posts.json` every time a post is created, edited, or deleted. To keep that logic in one place, `init.js` and `post.js` both import a single helper from `html.js`: ```js // html.js export function writeSiteFiles(posts) { rmSync("site", { recursive: true, force: true }); mkdirSync("site/posts", { recursive: true }); writeFileSync("site/index.html", generateIndex(posts)); for (const post of posts) { writeFileSync(`site/posts/${post.slug}.html`, generatePost(post)); } } function generateIndex(posts) { const items = posts .sort((a, b) => new Date(b.date) - new Date(a.date)) .map( (p) => `
${esc(p.title)} ${p.date}
` ) .join("\n"); return ` My Blog My Blog ${posts.length} post${posts.length !== 1 ? "s" : ""} ${items || "No posts yet."} `; } function generatePost(post) { return ` ${esc(post.title)} ← Back ${esc(post.title)} ${post.date} ${esc(post.body)} `; } function esc(s) { return s .replace(/&/g, "&") .replace(//g, ">") .replace(/"/g, """); } ``` `writeSiteFiles(posts)` wipes the local `site/` directory and rebuilds it from the array of posts. The Swarm-side logic — uploading the regenerated directory and pointing the feed at the new reference — lives in `init.js` and `post.js`. ### Initialize the Blog Create `init.js` — this generates a publisher key, creates an empty blog, uploads it, sets up the feed, and saves the configuration: ```js config(); const bee = new Bee(process.env.BEE_URL); const batchId = process.env.BATCH_ID; // 1. Generate publisher key const hex = "0x" + crypto.randomBytes(32).toString("hex"); const pk = new PrivateKey(hex); const owner = pk.publicKey().address(); const topic = Topic.fromString("blog"); // 2. Create initial empty blog const posts = []; writeFileSync("posts.json", JSON.stringify(posts, null, 2)); writeSiteFiles(posts); const upload = await bee.uploadFilesFromDirectory(batchId, "./site", { indexDocument: "index.html", }); // 3. Set up feed and manifest const writer = bee.makeFeedWriter(topic, pk); await writer.upload(batchId, upload.reference); const manifest = await bee.createFeedManifest(batchId, topic, owner); // 4. Save config const cfg = { privateKey: pk.toHex(), owner: owner.toHex(), topic: "blog", manifest: manifest.toHex(), }; writeFileSync("config.json", JSON.stringify(cfg, null, 2)); console.log("Blog initialized!"); console.log(`View your blog: ${process.env.BEE_URL}/bzz/${manifest.toHex()}/`); ``` Run it once: ```bash node init.js ``` Example output: ``` Blog initialized! Feed manifest: caa414d70028d14b0bdd9cbab18d1c1a0a3bab1b... View your blog: http://localhost:1633/bzz/caa414d70028d14b.../ ``` ### Create, Edit, and Delete Posts Create `post.js` — a single script that handles creating, editing, and deleting posts. Each operation modifies the local `posts.json`, regenerates the entire site, uploads it, and updates the feed: ```js config(); const [action, ...args] = process.argv.slice(2); if (!action || !["create", "edit", "delete"].includes(action)) { console.log(`Usage: node post.js create "" "<body>" node post.js edit <slug> "<title>" "<body>" node post.js delete <slug>`); process.exit(1); } const bee = new Bee(process.env.BEE_URL); const batchId = process.env.BATCH_ID; const cfg = JSON.parse(readFileSync("config.json", "utf-8")); const posts = JSON.parse(readFileSync("posts.json", "utf-8")); // --- Apply the action --- if (action === "create") { const [slug, title, body] = args; if (!slug || !title || !body) { console.error('Usage: node post.js create <slug> "<title>" "<body>"'); process.exit(1); } if (posts.find((p) => p.slug === slug)) { console.error(`Post "${slug}" already exists. Use "edit" to update it.`); process.exit(1); } posts.push({ slug, title, body, date: new Date().toISOString() }); console.log(`Created post: ${slug}`); } if (action === "edit") { const [slug, title, body] = args; if (!slug || !title || !body) { console.error('Usage: node post.js edit <slug> "<title>" "<body>"'); process.exit(1); } const idx = posts.findIndex((p) => p.slug === slug); if (idx === -1) { console.error(`Post "${slug}" not found.`); process.exit(1); } posts[idx] = { ...posts[idx], title, body, date: new Date().toISOString() }; console.log(`Edited post: ${slug}`); } if (action === "delete") { const [slug] = args; if (!slug) { console.error("Usage: node post.js delete <slug>"); process.exit(1); } const idx = posts.findIndex((p) => p.slug === slug); if (idx === -1) { console.error(`Post "${slug}" not found.`); process.exit(1); } posts.splice(idx, 1); console.log(`Deleted post: ${slug}`); } // --- Save, regenerate, upload, update feed --- writeFileSync("posts.json", JSON.stringify(posts, null, 2)); writeSiteFiles(posts); const pk = new PrivateKey(cfg.privateKey); const topic = Topic.fromString(cfg.topic); const writer = bee.makeFeedWriter(topic, pk); const upload = await bee.uploadFilesFromDirectory(batchId, "./site", { indexDocument: "index.html", }); await writer.upload(batchId, upload.reference); console.log(`Blog updated! (${posts.length} post${posts.length !== 1 ? "s" : ""})`); console.log(`View: ${process.env.BEE_URL}/bzz/${cfg.manifest}/`); ``` Usage: ```bash # Create a post node post.js create hello-world "Hello World" "This is my first blog post on Swarm." # Create another post node post.js create feeds-intro "Understanding Feeds" "Feeds provide mutable pointers on immutable storage." # Edit a post node post.js edit hello-world "Hello Swarm!" "Updated: this is my first post, now improved." # Delete a post node post.js delete feeds-intro ``` Each command regenerates the entire site and updates the feed. The same feed manifest URL always serves the latest version of the blog. :::tip Notice that editing and deleting work the same way as creating — modify the local data, regenerate the site, re-upload, update the feed. Swarm itself doesn't have "edit" or "delete" operations. The old versions of the site remain accessible via their direct Swarm hashes, but the feed manifest always resolves to the latest version. ::: ### Read the Feed This demonstrates how anyone can read the feed without the publisher's private key — only the owner address and topic (or the manifest hash) are needed: Create `read.js`: ```js config(); const bee = new Bee(process.env.BEE_URL); const cfg = JSON.parse(readFileSync("config.json", "utf-8")); const topic = Topic.fromString(cfg.topic); const owner = new EthAddress(cfg.owner); const reader = bee.makeFeedReader(topic, owner); const result = await reader.downloadReference(); console.log("Latest content reference:", result.reference.toHex()); console.log("Feed index:", result.feedIndex.toBigInt()); console.log("View:", `${process.env.BEE_URL}/bzz/${cfg.manifest}/`); ``` Run it: ```bash node read.js ``` ## Summary Feeds add a mutable pointer layer on top of Swarm's immutable storage. The core pattern is: upload content → write its reference to a feed → use a feed manifest as a stable URL. The same manifest URL always serves the latest content. This is the same pattern that [Etherjot](https://github.com/ethersphere/etherjot) uses to power fully decentralized blogs — regenerate the site, re-upload, and update the feed. The difference is only in scale and features (markdown rendering, categories, media management), not in the underlying feed mechanics. The `simple-blog` project you just built is a single-author blog. The next guide extends this into a multi-author system where each author controls their own feed and an admin maintains an index feed that links them all together. Key takeaways: - Every upload to Swarm is immutable and produces a unique hash. - A feed is a sequence of updates identified by an **owner** and a **topic**. - Each feed update stores a Swarm reference pointing to your content. - A **feed manifest** wraps the feed identity into a single permanent hash that resolves through `/bzz/`. - Only the feed owner (holder of the private key) can publish updates, but anyone can read the feed. - "Editing" and "deleting" content on Swarm means regenerating your site without the removed or changed content, re-uploading, and updating the feed. Old versions remain on Swarm at their original hashes, but the feed always points to the latest. --- **Next:** [Multi-Author Blog](/docs/develop/multi-author-blog) — extend feeds into a multi-publisher system where each author controls their own feed and a shared index links them together. --- ## Manage Files In the [Host a Webpage](/docs/develop/host-your-website) guide you uploaded a directory and got back a single Swarm reference that serves your site. That reference points to a **manifest** — a data structure that maps relative paths to content. This guide explores manifests directly: how to inspect them, add a file without re-uploading everything, and move a file by remapping a path. Swarm does not have a traditional filesystem — there are no mutable directories, in-place updates, or a built-in directory structure that preserves relationships between files. Instead, these capabilities are provided through the use of [manifests](./tools-and-features/manifests.md), which map relative paths (such as `/images/cat.jpg`) to immutable Swarm content references. When you upload a directory, Bee creates a manifest automatically and returns its reference. Files can then be accessed using paths that are relative to that manifest reference, based on the original directory structure. This provides filesystem-like behavior for your data, and the directory structure can later be changed by publishing a new version of the manifest with the desired updates. ## Usage and Example Scripts This section demonstrates how manifests enable filesystem-like features on Swarm, including uploading directories and modifying file paths. The full working scripts are available in the [examples](https://github.com/ethersphere/examples) repo: * [`script-01.js`](https://github.com/ethersphere/examples/blob/main/filesystem/script-01.js) * [`script-02.js`](https://github.com/ethersphere/examples/blob/main/filesystem/script-02.js) * [`script-03.js`](https://github.com/ethersphere/examples/blob/main/filesystem/script-03.js) :::info Website routing Manifests are also used for website routing (index documents, clean URLs, error pages, redirects). If you are building a website, see the [Routing guide](./routing.md). ::: ### Prerequisites * Node.js (v20+ recommended) * npm * A running Bee node (local or remote) * A funded postage batch Clone the [examples](https://github.com/ethersphere/examples) repo and navigate to the manifests directory: ```bash git clone https://github.com/ethersphere/examples.git cd examples/filesystem npm install ``` Copy `.env.example` to `.env` and fill in your values: ```bash cp .env.example .env ``` ```bash BEE_URL=http://localhost:1633 # or http://127.0.0.1:1633 BATCH_ID=<YOUR_BATCH_ID> UPLOAD_DIR=./folder SCRIPT_02_MANIFEST= SCRIPT_03_MANIFEST= ``` ## Script 1: Upload Folder and Inspect Manifest In this example, we simply upload a folder and print its manifest in a human readable format. Full script: * [`script-01.js`](https://github.com/ethersphere/examples/blob/main/filesystem/script-01.js) :::info Uploading is handled by a utility script: * [`upload-directory.js`](https://github.com/ethersphere/examples/blob/main/utils/upload-directory.js) The script: * Uploads a directory using `bee.uploadFilesFromDirectory` * Returns the manifest reference and prints it to the terminal The directory upload utility script itself looks like this: ```js const { reference } = await bee.uploadFilesFromDirectory(batchId, path, options); ``` The returned `reference` is for the **manifest itself**, not a file reference. Files must always be accessed *through* this manifest, not directly through file references shown in the manifest. ::: Run the script: ```bash node script-01.js ``` Script terminal output: ```bash [dotenv@17.2.3] injecting env (3) from .env -- tip: ⚙️ override existing env vars with { override: true } Uploaded directory: C:\Users\username\Documents\examples\filesystem\folder Reference: http://127.0.0.1:1633/bzz/bf5fa30cf426fe9b646db8cb1dfcb8fd146096e6a86c1de2b266689346e703c8 Manifest reference: bf5fa30cf426fe9b646db8cb1dfcb8fd146096e6a86c1de2b266689346e703c8 root.txt: ROOT DIRECTORY subfolder/nested.txt: NESTED DIRECTORY --- Manifest Tree --- { "path": "", "target": "0x0000000000000000000000000000000000000000000000000000000000000000", "metadata": null, "forks": { "/": { "path": "/", "target": "0x0000000000000000000000000000000000000000000000000000000000000000", "metadata": { "website-index-document": "disc.jpg" }, "forks": {} }, "disc.jpg": { "path": "disc.jpg", "target": "0xc4df63219e294cf412b4ad77169c8c6a30077af1b4160c3db6d536fdb7cc91df", "metadata": { "Content-Type": "image/jpeg", "Filename": "disc.jpg" }, "forks": {} }, "root.txt": { "path": "root.txt", "target": "0x45b3c65f9bcba9150247878baf9120836a51e62f61f7397270227a71ed94bfaf", "metadata": { "Content-Type": "text/plain; charset=utf-8", "Filename": "root.txt" }, "forks": {} }, "subfolder/nested.txt": { "path": "subfolder/nested.txt", "target": "0x7ca0eb93e9b5802fa5c62ca8e2ef84fffa73a0f589ef68fc457beccbb2b1f84f", "metadata": { "Content-Type": "text/plain; charset=utf-8", "Filename": "subfolder\\nested.txt" }, "forks": {} } } } ``` :::info Note that the manifest contains an entry for the file we specified as the index document in the upload options `indexDocument: "disc.jpg"` Both `indexDocument` and `errorDocument` options will cause the manifest to be updated, but for more complex manifest manipulation we will need to do a bit more than setting some options. Keep reading to learn how. ::: In the example output, you will find the following line (with your own unique manifest reference): ```bash Manifest reference: bf5fa30cf426fe9b646db8cb1dfcb8fd146096e6a86c1de2b266689346e703c8 ``` Update `SCRIPT_02_MANIFEST` in your `.env` file with the printed **manifest reference**: ```bash SCRIPT_02_MANIFEST=bf5fa30cf426fe9b646db8cb1dfcb8fd146096e6a86c1de2b266689346e703c8 ``` You will also see a formatted URL in the output: ```bash URL: http://localhost:1633/bzz/bf5fa30cf426fe9b646db8cb1dfcb8fd146096e6a86c1de2b266689346e703c8/ ``` Copy it and open in your browser. Since we specified an index document in our code, that's what you will see: Index document option: ```js indexDocument: "disc.jpg" ``` */img/disc.jpg:* ![DISC diagram](/img/disc.jpg) Since we DID NOT specify the `errorDocument` option, If you navigate to a non-existing document, you will just see your browser's default 404 error page: [http://localhost:1633/bzz/bf5fa30cf426fe9b646db8cb1dfcb8fd146096e6a86c1de2b266689346e703c8/non-existing-content](http://localhost:1633/bzz/bf5fa30cf426fe9b646db8cb1dfcb8fd146096e6a86c1de2b266689346e703c8/non-existing-content) ![browser default 404 page](/img/default-404.jpg) We'll fix this later. ### Code Explanation 1. Get path First we get the path to our upload directory as specified in the `.env` file by the `UPLOAD_DIR` variable: ```bash const directoryPath = path.join(__dirname, process.env.UPLOAD_DIR); ``` 2. Upload Then we upload the directory using our imported `uploadDirectory` utility function and set the index document to the "disc.jpg" in the root of our folder. Upon successful upload, the manifest reference is saved in `reference` and printed to the terminal: ```bash const reference = await uploadDirectory(directoryPath, { indexDocument: "disc.jpg" }); console.log("Manifest reference:", reference.toHex()); ``` 3. Print manifest After upload, the manifest is loaded and printed: ```js const node = await MantarayNode.unmarshal(bee, reference) await node.loadRecursively(bee) printManifestJson(node) ``` This produces a tree showing how paths map to Swarm references. To better understand the tree shown in the terminal output, refer to the [Manifests](./tools-and-features/manifests.md) page. ```bash "/": { "path": "/", "target": "0x0000000000000000000000000000000000000000000000000000000000000000", "metadata": { "website-index-document": "disc.jpg" }, "forks": {} }, ``` This entry ensures that a file will be served at the root directory rather than a 404 error. In the next script, we see how to update the manifest tree. ## Script 2: Adding a File to an Existing Manifest The second script demonstrates how to add a new file without re-uploading the entire directory. Full script: * [`script-02.js`](https://github.com/ethersphere/examples/blob/main/filesystem/script-02.js) :::tip Before running the second script, make sure that you have updated your `.env` variable `SCRIPT_02_MANIFEST` with the manifest reference returned by the first script. ::: ```bash node script-02.js ``` The terminal output will be similar to that from our first script except with several key differences: 1. Updated manifest reference Since we've updated the manifest, we now have a new manifest reference: ```bash Updated manifest reference: aaec0f55d6e9216944246f5adce0834c69b55ac2164ea1f5777dadf545b8f3bc Updated manifest URL: http://localhost:1633/bzz/aaec0f55d6e9216944246f5adce0834c69b55ac2164ea1f5777dadf545b8f3bc/ ``` Update `SCRIPT_03_MANIFEST` in your `.env` file with the **Updated manifest reference**: ```bash SCRIPT_03_MANIFEST=aaec0f55d6e9216944246f5adce0834c69b55ac2164ea1f5777dadf545b8f3bc ``` 2. Modified directory tree ```bash "new.txt": { "path": "new.txt", "target": "0x3515db2f5e3c075b7546d7dd7dea1680c3e0785c6584e66b7e4f56fc344a0a78", "metadata": { "Content-Type": "text/plain; charset=utf-8", "Filename": "new.txt" }, "forks": {} } ``` Now if we navigate to [http://localhost:1633/bzz/aaec0f55d6e9216944246f5adce0834c69b55ac2164ea1f5777dadf545b8f3bc/new.txt](http://localhost:1633/bzz/aaec0f55d6e9216944246f5adce0834c69b55ac2164ea1f5777dadf545b8f3bc/new.txt), we will see the contents of the new file added to our manifest: ```bash Hi, I'm new here. ``` ### Explanation 1. Load the existing manifest returned from the first script: ```js const node = await MantarayNode.unmarshal(bee, ROOT_MANIFEST) await node.loadRecursively(bee) ``` 2. Upload a new file we intend to add to the manifest (not a directory): ```js const { reference } = await bee.uploadData(batchId, bytes) ``` 3. Insert the file into the manifest: ```js node.addFork(filename, reference, metadata) ``` 4. Save the updated manifest: ```js const updated = await node.saveRecursively(bee, batchId) ``` This produces a **new manifest reference** where the file is now accessible by path, for example: ```bash swarm-cli download aaec0f55d6e9216944246f5adce0834c69b55ac2164ea1f5777dadf545b8f3bc/new.txt ./ new.txt OK ``` Print file contents to confirm: ```bash cat .\new.txt Hi, I'm new here. ``` Our new file is now accessible through the same manifest reference along with all our other files. ## Script 3: Moving a File by Updating the Manifest The third script shows how to move a file by modifying paths in the manifest. Full script: * [`script-03.js`](https://github.com/ethersphere/examples/blob/main/filesystem/script-03.js) :::tip Before running the third script, make sure that you have updated your `.env` variable `SCRIPT_03_MANIFEST` with the manifest reference returned by the second script (see terminal output from `Updated manifest reference:`). ::: This is done by: 1. Locating the existing file entry 2. Removing it from its current path 3. Re-adding it under a new path Run the script: ```bash node script-03.js ``` The output should look familiar, but again with several key changes: 1. Updated manifest reference Since we've made another change to the manifest, we have a new manifest reference: ```bash Updated manifest reference: 9a4a6305c811b2976498ef38270fffeb16966fc8719f745a4b18598d39e77ae0 ``` 2. Modified directory tree We no longer see the entry for `new.txt` at the root directory, and we now have a new entry for the same file but now at an updated path in a nested directory: ```bash "nested/deeper/new.txt": { "path": "nested/deeper/new.txt", "target": "0x3515db2f5e3c075b7546d7dd7dea1680c3e0785c6584e66b7e4f56fc344a0a78", "metadata": { "Content-Type": "text/plain; charset=utf-8", "Filename": "new.txt" }, "forks": {} } ``` If we navigate to the `new.txt` file in its old location: [http://localhost:1633/bzz/9a4a6305c811b2976498ef38270fffeb16966fc8719f745a4b18598d39e77ae0/new.txt](http://localhost:1633/bzz/9a4a6305c811b2976498ef38270fffeb16966fc8719f745a4b18598d39e77ae0/new.txt) We will get a 404 error since we removed that entry from the manifest. But if we navigate to its new location at `/nested/deeper/new.txt` we will now see it again: [http://localhost:1633/bzz/9a4a6305c811b2976498ef38270fffeb16966fc8719f745a4b18598d39e77ae0/nested/deeper/new.txt](http://localhost:1633/bzz/9a4a6305c811b2976498ef38270fffeb16966fc8719f745a4b18598d39e77ae0/nested/deeper/new.txt) ```bash Hi, I'm new here. ``` ### Explanation 1. Remove entry Remove the entry for `new.txt` which was added by the second script: ```js node.removeFork("new.txt") ``` 2. Add new entry Add a new entry for `new.txt` in a new location in a nested directory: ```js node.addFork( "nested/deeper/new.txt", fileRef, metadata ) ``` 3. Save and print ```bash const updated = await node.saveRecursively(bee, batchId); const newManifestRef = updated.reference.toHex(); ``` After saving the manifest again, the file becomes accessible at: ``` /nested/deeper/new.txt ``` No data is duplicated, the `new.txt` file has not been modified, only the path mapping changes in the manifest. ## Key Takeaways * Uploading a directory creates a manifest * Files are accessed via the manifest, not directly by their internal references * Manifests can be modified to add, move, or remove files * Updating a manifest produces a new reference, but underlying data remains immutable * This provides filesystem-like behavior without mutable storage With these tools, you can treat Swarm directories much like a filesystem — while still preserving immutability and content addressing. --- **Next:** [Website Routing](/docs/develop/routing) — put manifest path mapping to practical use by setting up clean URL routing for a Swarm-hosted site. --- ## Run a Gateway A Swarm gateway is an HTTP server that makes Swarm-hosted websites reachable from an ordinary web browser, without visitors needing to run their own Bee node. This guide shows how to run your Bee node as a public HTTP gateway. This guide explains how to use the [swarm-gateway](https://github.com/ethersphere/swarm-gateway) tool to set up your node in gateway mode. Running your node in gateway mode exposes it publicly, allowing access through any typical browser or http API. It is divided into several parts: * Part 1 - Basic setup * Part 2 - Securing your gateway with TLS * Part 3 - Optional features ## Part 1 — Running a Swarm Gateway (HTTP, minimal setup) :::info Historically, the main tool for running a Swarm HTTP gateway was [gateway-proxy](https://github.com/ethersphere/gateway-proxy), however it is planned to be deprecated in favor of [swarm-gateway](https://github.com/ethersphere/swarm-gateway). At the time of writing, `gateway-proxy` still contains some features that are not yet implemented in `swarm-gateway` - unless you have a specific need for these features however, `swarm-gateway` is strongly recommended. ::: This guide describes how to run a Swarm HTTP gateway using `swarm-gateway` and Bee with a minimal configuration. At the end of this section, the gateway will be reachable at: ```text http://your-domain.example ``` Swarm content will be accessible at: ```text http://your-domain.example/bzz/<reference>/ ``` :::warning Security notice This setup uses plain HTTP. Traffic is not encrypted and any `Authorization` headers can be observed by intermediaries on the network path. This configuration is not suitable for production use. The purpose of this section is to verify that the gateway is working. HTTPS is added in a later part of the guide. ::: The guide in this section: * Runs `swarm-gateway` using Docker * Connects it to an existing Bee node * Exposes it publicly over HTTP :::danger This part of the guide does not cover setting up TLS, so your gateway will be accessible through plain HTTP, not HTTPS, making it highly insecure. It should not be exposed publicly without first setting up TLS, which is covered in the next section. ::: ### Prerequisites * A VPS with: * A public IP address * Port **80** open * Docker * A domain for hosting your gateway publicly * A running Bee node in Docker * A valid stamp batch ### 1. Configure DNS for your domain Create an A record in your DNS provider pointing your domain to your server's IP address: ```text your-domain.example -> <your-server-ip> ``` After DNS propagation, verify that the domain resolves to your server (this may take some time, to verify more quickly, try pinging from a different machine or VPS): ```bash ping your-domain.example ``` ### 2. Create a Docker network The gateway container must be able to communicate with your Bee node, for this, both containers must be on the same Docker network. Create a network and attach the Bee container to it: ```bash docker network create swarm-net docker network connect swarm-net bee-1 ``` Verify: ```bash docker network inspect swarm-net ``` The output should list `bee-1` as an attached container. ### 3. Pull the gateway image ```bash docker pull ethersphere/swarm-gateway:0.1.6 ``` ### 4. Run the gateway Start the gateway container: ```bash docker run -d --restart unless-stopped \ --name swarm-gateway \ --network swarm-net \ -p 80:3000 \ -e HOSTNAME="your-domain.example" \ -e BEE_API_URL="http://bee-1:1633" \ -e DATABASE_CONFIG="{}" \ ethersphere/swarm-gateway:0.1.6 ``` In this configuration, database-backed features such as subdomain rewrites and moderation are not configured. ### 5. Verify operation From your local machine (not the server on your VPS): ```bash curl http://your-domain.example/health ``` Expected output: ```text OK ``` ### 6. Test with existing content To confirm the gateway is correctly serving content from Swarm, request a reference that is already on the network. The following hash points to a small JSON file: ```text http://your-domain.example/bzz/f3f5e25c90824876c2468b9bdf0d842cd05dc5f0974681789b9729bc155c4f65/ ``` Expected output: ```json { "octalmage.com": "bzz://45f0f1e13b70e2919e59fdc5bcf3a99bcbe19dc1be6ebdebe3f89794b77c19ab/", "o8.is": "bzz://4cd43b1c0ebc257f79cc45ebd9774e1251e34f08026325c78ef2ca46972935cc/", "dist.o8.is": "bzz://0890110b61109aee2b6f0d071cedce584868bb29dcb7e41b1c0388d6cf775ace/" } ``` If the JSON is returned, your gateway is correctly fetching and serving content from Swarm. To serve your own content, upload a file or website through your Bee node (see the [Upload and Download](/docs/develop/upload-and-download) and [Host a Webpage](/docs/develop/host-your-website) guides) and use the resulting reference in place of the one above. ### 7. Optional: restrict uploads using authentication By default, the gateway allows anyone to upload content using your Bee node. To restrict uploads, set: * `AUTH_SECRET` — a long random string * `SOFT_AUTH=true` — only require authentication for POST requests Example (add these lines to the `docker run` command): ```bash -e AUTH_SECRET="replace-with-a-long-random-secret" \ -e SOFT_AUTH="true" \ ``` A minimal gateway setup consists of: * A working Swarm HTTP gateway * Connected to your Bee node * Exposing content publicly over `/bzz/<reference>` The setup is intentionally minimal and suitable for testing and development, however without TLS, it is not secure and should never be used in production or publicly exposed. The next section explains how to enable TLS so that your gateway can be securely accessed through HTTPS. ## Part 2 — Securing your gateway with TLS (HTTPS) This section explains how to secure your gateway using **TLS (HTTPS)** with **Caddy**. Caddy is used here as a front-facing web server that automatically manages TLS certificates and forwards traffic to `swarm-gateway`. In this setup: * Caddy is responsible only for HTTPS and certificate management * `swarm-gateway` continues to act as the application gateway and reverse proxy for Bee At the end of this section, your gateway will be reachable at: ```text https://your-domain.example ``` And all HTTP traffic will be automatically redirected to HTTPS. ### Prerequisites In addition to the prerequisites from Part 1: * Your domain must already point to your VPS IP address * Ports **80** and **443** must be open on your VPS firewall ### 1. Reconfigure the gateway to not expose port 80 Caddy will become the public entry point, so `swarm-gateway` should no longer be exposed directly. Stop and remove the existing container: ```bash docker stop swarm-gateway docker rm swarm-gateway ``` Recreate it **without** publishing port 80: ```bash docker run -d --restart unless-stopped \ --name swarm-gateway \ --network swarm-net \ -e HOSTNAME="your-domain.example" \ -e BEE_API_URL="http://bee-1:1633" \ -e DATABASE_CONFIG="{}" \ ethersphere/swarm-gateway:0.1.6 ``` The gateway is now only accessible from within the Docker network. ### 2. Create a Caddy configuration Create a directory for the Caddy configuration: ```bash mkdir -p ~/caddy cd ~/caddy ``` Create a file named `Caddyfile`: ```bash nano Caddyfile ``` Add the following configuration (replace the domain): ```caddy your-domain.example { reverse_proxy swarm-gateway:3000 } ``` ### 3. Run Caddy Start Caddy in Docker and attach it to the same Docker network: ```bash docker run -d --restart unless-stopped \ --name caddy \ --network swarm-net \ -p 80:80 \ -p 443:443 \ -v $HOME/caddy/Caddyfile:/etc/caddy/Caddyfile \ -v caddy_data:/data \ -v caddy_config:/config \ caddy:2 ``` Caddy will automatically: * Obtain a TLS certificate for your domain * Renew it before it expires * Redirect all HTTP traffic to HTTPS ### 4. Verify operation From your local machine: ```bash curl https://your-domain.example/health ``` Expected output: ```text OK ``` You can also verify that HTTP is redirected to HTTPS: ```bash curl -I http://your-domain.example/health ``` --- **Next:** [Dynamic Content](/docs/develop/dynamic-content) — return to app development and learn how feeds add a mutable pointer layer on top of Swarm's immutable storage. --- ## Host a Webpage In the [Upload and Download](/docs/develop/upload-and-download) guide you uploaded individual files and got back Swarm reference hashes. A website is just a collection of files — an HTML page, a stylesheet, maybe an image. When you upload a directory, Bee automatically builds a [manifest](/docs/develop/tools-and-features/manifests) that maps each relative path to its content. Set an `indexDocument` and the root URL resolves to your homepage. This guide shows how to upload a static site and open it through `/bzz/<reference>/`. :::info Example project The example website used in this guide is in [`examples/website`](https://github.com/ethersphere/examples/tree/main/website). Clone the repo, copy `.env.example` to `.env`, fill in your values, and run `npm install && npm run upload`. ::: ## Prerequisites * A running Bee node (either a [standard installation](./../bee/installation/quick-start.md) or [Swarm Desktop](./../desktop/install.md)) * A valid postage stamp batch * Node.js (18+) and `@ethersphere/bee-js` installed in your project * Static website files (HTML, CSS, etc.) — feel free to use the [provided example site](https://github.com/ethersphere/examples/tree/main/website) ## Upload and Access by Hash Install bee-js: ```bash npm install @ethersphere/bee-js ``` Website upload script: ```js const bee = new Bee("http://localhost:1633"); const batchId = "<BATCH_ID>"; // Replace with your actual postage batch ID const result = await bee.uploadFilesFromDirectory(batchId, "./website", { indexDocument: "index.html", errorDocument: "404.html" }); console.log("Swarm hash:", result.reference.toHex()); ``` ```bash Swarm hash: 6c45eae389b3bffce21443316d0bd47c4101545092b7c72c313a33ee7d003475 ``` After running the script, copy the Swarm hash output to the console and then use it to open your Swarm hosted website in the browser: ```bash http://localhost:1633/bzz/<SWARM_HASH>/ ``` ## Advanced: Keep Your URL Stable Across Updates :::note Prerequisite This section uses feeds — the concept of a mutable pointer on top of Swarm's immutable storage. Feeds are explained from scratch in the [Dynamic Content](/docs/develop/dynamic-content) guide. You can complete this guide without this section and come back after you have worked through Dynamic Content. ::: Every time you re-upload a site to Swarm, you get a new reference hash. If you want a single stable URL that always points to the latest version of your site — useful for ENS integration or sharing a permanent link — publish each upload as a feed entry and share the feed manifest hash instead of the content hash. :::tip You will need a publisher key to use for setting up your website feed. You can use the `PrivateKey` class to generate a dedicated publisher key: ```js const crypto = require('crypto'); const { PrivateKey } = require('@ethersphere/bee-js'); // Generate 32 random bytes and construct a private key const hexKey = '0x' + crypto.randomBytes(32).toString('hex'); const privateKey = new PrivateKey(hexKey); console.log('Private key:', privateKey.toHex()); console.log('Public address:', privateKey.publicKey().address().toHex()); ```` Example output: ```bash Private key: 634fb5a872396d9693e5c9f9d7233cfa93f395c093371017ff44aa9ae6564cdd Public address: 8d3766440f0d7b949a5e32995d09619a7f86e632 ``` Store this key securely. Anyone with access to it can publish to your feed. *It is recommended to use a separate publishing key for each feed.* ::: ### Example Script :::tip The script below refers to some core feed concepts such as the feed "topic" and "writer". To learn more about these concepts and feeds in general, refer to the [bee-js documentation](https://bee-js.ethswarm.org/docs/soc-and-feeds/#feeds). ::: The script performs these steps: 1. **Connects to your Bee node** and loads your postage batch + publisher private key. 2. **Creates a feed topic and writer** for publishing website updates. 3. **Uploads the `./website` directory** to Swarm and logs the resulting content hash. 4. **Publishes that hash to the feed** so it becomes the latest feed entry. 5. **Creates a feed manifest** and logs its reference — this is the permanent hash you use for ENS or stable URLs. ```js const bee = new Bee("http://localhost:1633"); const batchId = "<BATCH_ID>" // Replace with your batch id const privateKey = new PrivateKey("<PUBLISHER_KEY>"); // Replace with your publisher private key const owner = privateKey.publicKey().address(); // Upload and Create Feed Manifest const topic = Topic.fromString("website"); const writer = bee.makeFeedWriter(topic, privateKey); const upload = await bee.uploadFilesFromDirectory(batchId, "./website", { indexDocument: "index.html", errorDocument: "404.html" }); console.log("Website Swarm Hash:", upload.reference.toHex()) await writer.uploadReference(batchId, upload.reference); const manifestRef = await bee.createFeedManifest(batchId, topic, owner); console.log("Feed Manifest:", manifestRef.toHex()); ``` Upon the successful execution of the script, the hash of the uploaded website will be logged along feed manifest hash. Copy the "Feed Manifest" hash to be used in the next step: ```bash Website Swarm Hash: 6c45eae389b3bffce21443316d0bd47c4101545092b7c72c313a33ee7d003475 Feed Manifest: caa414d70028d14b0bdd9cbab18d1c1a0a3bab1b20a56cf06937a6b20c7e7377 ``` Follow the [official ENS guide](https://support.ens.domains/en/articles/12275979-how-do-i-add-a-decentralised-website-to-my-ens-name) for registering a content hash adding your content hash in the ENS UI (see [guide](#optional-connect-site-to-ens-domain)). However, rather than registering your website's hash directly, register the feed manifest hash we saved from the previous step from our example above. ``` bzz://<manifestRef> ``` Future updates just re-run: ```js await writer.upload(batchId, newUpload.reference); ``` Your ENS domain will always point to the latest upload via the feed manifest. You’ve now got a programmatic way to deploy and update your Swarm-hosted site with ENS support using `bee-js`! ## Optional: Connect Site to ENS Domain Once your site is uploaded to Swarm, you can make it accessible via an easy to remember ENS domain name rather than its Swarm hash: ``` https://yourname.eth.limo/ https://yourname.bzz.link/ ``` or through your own node: ``` http://localhost:1633/bzz/yourname.eth/ ``` ### Using the Official ENS Guide ENS provides a clear walkthrough with screenshots showing how to add a content hash to your domain with their [easy to use app](https://app.ens.domains/): [How to add a Decentralized website to an ENS name](https://support.ens.domains/en/articles/12275979-how-do-i-add-a-decentralised-website-to-my-ens-name) The guide covers: * Opening your ENS domain in the ENS Manager * Navigating to the Records tab * Adding a Content Hash * Confirming the transaction ### Swarm-Specific Step When you reach Step 2 in the ENS guide (“Add content hash record”), enter your Swarm reference in the following format: :::tip For the content hash, you can use a Swarm-hosted website's hash directly, or — as recommended in the [Advanced: Keep Your URL Stable Across Updates](#advanced-keep-your-url-stable-across-updates) section above — publish your site to a feed and use the feed manifest hash instead. By using a feed manifest as the content hash, you can avoid repeated ENS registry updates. ::: :::tip If ENS does not resolve on localhost If the site doesn't load from `http://localhost:1633/bzz/yourname.eth/`, the issue is usually the ENS resolver RPC. Free public endpoints like `https://cloudflare-eth.com` [may not resolve reliably](https://developers.cloudflare.com/web3/reference/migration-guide/?utm_source=chatgpt.com). Reliable alternatives include `https://mainnet.infura.io/v3/<infura-api-key>` and `https://eth-mainnet.public.blastapi.io`, or run your own Ethereum node. ::: ``` bzz://<SWARM_HASH> ``` Example: ``` bzz://cf50756e6115445fd283691673fa4ad2204849558a6f3b3f4e632440f1c3ab7c ``` This works across: * eth.limo and bzz.link * localhost (with a compatible RPC) * any ENS-compatible Swarm resolver You do not need to encode the hash or use any additional tools. `bzz://<hash>` is sufficient. --- **Next:** [Manage Files](/docs/develop/files) — learn how manifests provide filesystem-like path mapping and how to add, move, or remove files without re-uploading everything. --- ## Building on Swarm This is the go-to starting point for web3 developers who want to build with Swarm. The guides on this page will help you get started with setting up a Bee node, using that node to integrate your dApp with Swarm, and to begin exploring some example applications to better understand the possibilities of building on Swarm. ## Setup Install Bee Install and run a Bee node — your entry point for development on the Swarm network. Get started Connect Your App Use the official <code>bee-js</code> SDK to connect your app to your Bee node and integrate Swarm based storage, feeds, and more. Connect dApp Onboard with AI Skills Use the Swarm Quickstart Skills in Claude Code — type <code>/swarm</code> for guided, prerequisite-checking steps to set up a node and start building. Start with AI Swarm Cheatsheet A dense, printable quick-reference for building on Swarm — what it is, its limits, and curated links to get started fast. View cheatsheet ## Guides Upload and Download Learn how to upload and download a variety of data types from Swarm (with support for both browser and Node.js environments). Open guide Host a Webpage Host a website on Swarm and link it to your ENS domain for easy access at through gateways and Bee nodes. Open guide Manage Files Learn about how manifests enable a virtual "filesystem" on Swarm, and how to manipulate the manifest to re-write virtual paths to add, remove, or move content. Open Guide Website Routing Learn about routing on Swarm and the various options at your disposal for approaching website routing. Open Guide Run a Gateway Run your own Swarm HTTP gateway to serve content from the network and make it accessible to browsers and other HTTP clients. Open guide Dynamic Content Learn how to use feeds to create updatable content on Swarm — with a complete example project that builds a dynamic note board. Open guide Multi-Author Blog Build a decentralized multi-author blog where each author controls their own feed and an admin index feed links them all together. Open guide <!-- Add Access Control Control who is able to view your uploaded content, such as a list of paid newsletter or blog subscribers. Open guide Messaging on Swarm Learn how to integrate Swarm-based messaging into your app — with support for both real-time messaging use cases and private encrypted messaging. Open guide --> --- ## Multi-Author Blog This guide extends the [Dynamic Content](/docs/develop/dynamic-content) pattern into a multi-author system. Instead of a single publisher managing one feed, each author independently controls their own feed, and an admin maintains an index feed that links them all together. This demonstrates the core architectural pattern needed for decentralized networks: **feeds that reference other feeds**. The key insight is that a feed entry does not have to point to HTML content — it can point to any Swarm data, including a JSON manifest that describes other feeds. This creates a composable, decentralized publishing network without any central coordinator beyond a shared index feed. ## Prerequisites * A running Bee node ([install guide](/docs/bee/installation/quick-start)) * A valid postage stamp batch ([how to get one](/docs/develop/tools-and-features/buy-a-stamp-batch)) * Node.js 18+ and `@ethersphere/bee-js` installed * Familiarity with the [Dynamic Content](/docs/develop/dynamic-content) guide and feeds ## Architecture The multi-author blog consists of four feed layers: ``` Index Feed (admin key, topic: "blog-index") └─ points to → authors.json ├─ { name: "Alice", topic: "alice-posts", owner: "0xAlice...", feedManifest: "3fa19c..." } └─ { name: "Bob", topic: "bob-posts", owner: "0xBob...", feedManifest: "7c244b..." } Alice's Feed (alice key, topic: "alice-posts") └─ points to → alice's blog page HTML Bob's Feed (bob key, topic: "bob-posts") └─ points to → bob's blog page HTML Homepage Feed (admin key, topic: "blog-home") └─ points to → index.html (aggregated view reading all author feeds) ``` Each author publishes independently to their own feed. The admin reads from all author feeds, assembles an aggregated homepage, and publishes it. The index feed stores the master list of authors — any new reader can discover all authors by reading the index. ## Feeds Referencing Feeds The Simple Blog example in the [Dynamic Content guide](/docs/develop/dynamic-content#example-project--simple-blog) demonstrated regenerate-and-publish: upload content, point a feed to it, update the feed manifest URL. The multi-author blog adds a new dimension: **feeds as data structures**. When you store a JSON document inside a feed that contains the `topic` and `owner` of other feeds, you've created a directory of feeds — a linked network. The `authors.json` file is not just content; it's a data structure that enumerates other feeds and their stable references (feed manifest hashes). :::tip A feed manifest hash is a stable, permanent reference to a feed. You can store feed manifest hashes inside your index feed's JSON payload, and readers just need that manifest hash to follow the link — they don't need the topic string or owner address separately. Manifest hashes are your "URLs" between feeds. ::: This pattern scales. You can have hundreds of author feeds, all discovered through a single index feed. Add a new author by appending their entry to `authors.json` and re-uploading to the index feed. Readers polling the index automatically discover the new author — no out-of-band notification needed. ## Example Scripts The complete project is in the [`multi-author-blog`](https://github.com/ethersphere/examples/tree/main/multi-author-blog) directory of the [examples](https://github.com/ethersphere/examples) repo: * [`init.js`](https://github.com/ethersphere/examples/blob/main/multi-author-blog/init.js) — One-time setup: create all feeds and manifests * [`add-post.js`](https://github.com/ethersphere/examples/blob/main/multi-author-blog/add-post.js) — Author publishes a new post * [`update-index.js`](https://github.com/ethersphere/examples/blob/main/multi-author-blog/update-index.js) — Admin aggregates author feeds and updates homepage * [`add-author.js`](https://github.com/ethersphere/examples/blob/main/multi-author-blog/add-author.js) — Add a new author to the blog * [`read.js`](https://github.com/ethersphere/examples/blob/main/multi-author-blog/read.js) — Read the feeds without private keys Clone the repo and set up the project: ```bash git clone https://github.com/ethersphere/examples.git cd examples/multi-author-blog npm install cp .env.example .env ``` Fill in your `BATCH_ID` in `.env`: ```bash BEE_URL=http://localhost:1633 BATCH_ID=<YOUR_BATCH_ID> ``` ## Example Project — Multi-Author Blog This section builds a complete runnable project: a blog where multiple authors publish independently, and an admin maintains a homepage that aggregates all posts. ### Project Setup Use the cloned examples repo (see [Example Scripts](#example-scripts) above): ```bash cd examples/multi-author-blog npm install cp .env.example .env # Fill in BEE_URL and BATCH_ID in .env ``` ### Project Structure ``` swarm-multiblog/ ├── .env ├── config.json # Created by init.js — all keys and manifest hashes ├── authors.json # Created by init.js — directory of authors ├── alice-posts.json # Created by add-post.js — Alice's post list ├── bob-posts.json # Created by add-post.js — Bob's post list ├── init.js # One-time setup: create all feeds and manifests ├── add-post.js # Author publishes a new post ├── update-index.js # Admin aggregates author feeds and updates homepage └── read.js # Read the feeds without private keys ``` ### Initialize the Blog This step generates keys for all authors and the admin, creates feeds for each author and for the homepage, builds the index feed with an `authors.json` manifest, and saves everything to `config.json`. Create `init.js`: ```js config(); const bee = new Bee(process.env.BEE_URL); const batchId = process.env.BATCH_ID; function makeKey() { const hex = "0x" + crypto.randomBytes(32).toString("hex"); return new PrivateKey(hex); } // Generate keys for admin, Alice, and Bob const adminKey = makeKey(); const aliceKey = makeKey(); const bobKey = makeKey(); const adminOwner = adminKey.publicKey().address(); const aliceOwner = aliceKey.publicKey().address(); const bobOwner = bobKey.publicKey().address(); // Topics — each feed has a unique topic const aliceTopic = Topic.fromString("alice-posts"); const bobTopic = Topic.fromString("bob-posts"); const indexTopic = Topic.fromString("blog-index"); const homeTopic = Topic.fromString("blog-home"); // --- Step 1: Upload initial author pages --- const aliceHTML = generateAuthorHTML("Alice", []); const bobHTML = generateAuthorHTML("Bob", []); const aliceUpload = await bee.uploadFile(batchId, aliceHTML, "index.html", { contentType: "text/html", }); const bobUpload = await bee.uploadFile(batchId, bobHTML, "index.html", { contentType: "text/html", }); // --- Step 2: Create author feeds --- const aliceWriter = bee.makeFeedWriter(aliceTopic, aliceKey); const bobWriter = bee.makeFeedWriter(bobTopic, bobKey); await aliceWriter.upload(batchId, aliceUpload.reference); await bobWriter.upload(batchId, bobUpload.reference); // --- Step 3: Create author feed manifests (stable references) --- const aliceManifest = await bee.createFeedManifest(batchId, aliceTopic, aliceOwner); const bobManifest = await bee.createFeedManifest(batchId, bobTopic, bobOwner); console.log("Alice feed manifest:", aliceManifest.toHex()); console.log("Bob feed manifest: ", bobManifest.toHex()); // --- Step 4: Build and upload the authors.json index --- const authors = [ { name: "Alice", topic: "alice-posts", owner: aliceOwner.toHex(), feedManifest: aliceManifest.toHex(), }, { name: "Bob", topic: "bob-posts", owner: bobOwner.toHex(), feedManifest: bobManifest.toHex(), }, ]; const authorsJson = JSON.stringify(authors, null, 2); writeFileSync("authors.json", authorsJson); const indexUpload = await bee.uploadFile(batchId, authorsJson, "authors.json", { contentType: "application/json", }); // --- Step 5: Create the index feed --- const indexWriter = bee.makeFeedWriter(indexTopic, adminKey); await indexWriter.upload(batchId, indexUpload.reference); const indexManifest = await bee.createFeedManifest(batchId, indexTopic, adminOwner); console.log("Index feed manifest:", indexManifest.toHex()); // --- Step 6: Generate and upload the homepage --- const homeHTML = generateHomepageHTML(authors, []); const homeUpload = await bee.uploadFile(batchId, homeHTML, "index.html", { contentType: "text/html", }); const homeWriter = bee.makeFeedWriter(homeTopic, adminKey); await homeWriter.upload(batchId, homeUpload.reference); const homeManifest = await bee.createFeedManifest(batchId, homeTopic, adminOwner); // --- Step 7: Save config --- const cfg = { admin: { privateKey: adminKey.toHex(), owner: adminOwner.toHex() }, alice: { privateKey: aliceKey.toHex(), owner: aliceOwner.toHex() }, bob: { privateKey: bobKey.toHex(), owner: bobOwner.toHex() }, topics: { alice: "alice-posts", bob: "bob-posts", index: "blog-index", home: "blog-home", }, manifests: { alice: aliceManifest.toHex(), bob: bobManifest.toHex(), index: indexManifest.toHex(), home: homeManifest.toHex(), }, }; writeFileSync("config.json", JSON.stringify(cfg, null, 2)); console.log("\nBlog initialized!"); console.log("Homepage: " + `${process.env.BEE_URL}/bzz/${homeManifest.toHex()}/`); console.log("Alice's feed: " + `${process.env.BEE_URL}/bzz/${aliceManifest.toHex()}/`); console.log("Bob's feed: " + `${process.env.BEE_URL}/bzz/${bobManifest.toHex()}/`); function generateAuthorHTML(name, posts) { const items = posts .map( (p) => ` ${p.title} <small style="color:#888;">${p.date}</small> ${p.body} ` ) .join("\n"); return `<!DOCTYPE html> <html> <head><meta charset="utf-8"><title>${name}'s Blog ${name}'s Blog ${posts.length} post${posts.length !== 1 ? "s" : ""} ${items || "No posts yet."} `; } function generateHomepageHTML(authors, latestPosts) { const cards = authors .map( (a) => { const latest = latestPosts.find((p) => p.author === a.name); const preview = latest ? `${latest.title} — ${latest.date}${latest.body.slice(0, 120)}…` : `No posts yet.`; return ` ${a.name} ${preview} `; } ) .join("\n"); return ` Multi-Author Blog Multi-Author Blog ${authors.length} author${authors.length !== 1 ? "s" : ""} ${cards || "No authors yet."} `; } ``` Run it once to initialize the blog: ```bash node init.js # or: npm run init ``` Example output: ``` Alice feed manifest: 3fa19c... Bob feed manifest: 7c244b... Index feed manifest: a10be5... Blog initialized! Homepage: http://localhost:1633/bzz/d991f2.../ Alice's feed: http://localhost:1633/bzz/3fa19c.../ Bob's feed: http://localhost:1633/bzz/7c244b.../ ``` ### Add a Post Authors publish independently. Each author regenerates their blog page with the new post, uploads it, and updates their feed. The admin can then aggregate the latest posts into the homepage. Create `add-post.js`: ```js config(); const [,, authorArg, title, ...bodyWords] = process.argv; const body = bodyWords.join(" "); if (!authorArg || !title || !body) { console.error('Usage: node add-post.js "Post title" "Post body"'); process.exit(1); } const bee = new Bee(process.env.BEE_URL); const batchId = process.env.BATCH_ID; const cfg = JSON.parse(readFileSync("config.json", "utf-8")); const author = cfg[authorArg]; if (!author) { console.error(`Unknown author: ${authorArg}`); process.exit(1); } const pk = new PrivateKey(author.privateKey); const topic = Topic.fromString(cfg.topics[authorArg]); // Load or initialize the author's post list const postsFile = `${authorArg}-posts.json`; let posts = []; try { posts = JSON.parse(readFileSync(postsFile, "utf-8")); } catch { // First post — file doesn't exist yet } const newPost = { title, body, date: new Date().toISOString() }; posts.push(newPost); writeFileSync(postsFile, JSON.stringify(posts, null, 2)); // Regenerate the author's page HTML const html = generateAuthorHTML( authorArg.charAt(0).toUpperCase() + authorArg.slice(1), posts ); // Upload and update the author's feed const upload = await bee.uploadFile(batchId, html, "index.html", { contentType: "text/html", }); const writer = bee.makeFeedWriter(topic, pk); await writer.upload(batchId, upload.reference); console.log(`Post published by ${authorArg}! (${posts.length} total)`); console.log("View: " + `${process.env.BEE_URL}/bzz/${cfg.manifests[authorArg]}/`); function generateAuthorHTML(name, posts) { const items = posts .map( (p) => ` ${p.title} ${p.date} ${p.body} ` ) .join("\n"); return ` ${name}'s Blog ${name}'s Blog ${posts.length} post${posts.length !== 1 ? "s" : ""} ${items} `; } ``` Run it: ```bash node add-post.js alice "Hello Swarm" "My first post on a decentralized blog." node add-post.js bob "Why Swarm?" "Censorship resistance matters." # or: npm run add-post -- alice "Hello Swarm" "My first post on a decentralized blog." ``` :::tip Authors are fully independent. Bob can publish a post without Alice's involvement, without any coordination, and without running any admin script. Each author controls only their own private key and topic. The admin (homepage aggregator) runs separately and at their own discretion. ::: ### Update the Homepage The admin aggregates all author feeds and publishes an updated homepage with previews of their latest posts. This is the key demonstration of feeds referencing feeds: the aggregator reads the index feed to discover authors, then reads each author's feed to fetch their latest content. Create `update-index.js`: ```js config(); const bee = new Bee(process.env.BEE_URL); const batchId = process.env.BATCH_ID; const cfg = JSON.parse(readFileSync("config.json", "utf-8")); const authors = JSON.parse(readFileSync("authors.json", "utf-8")); // Read each author's latest feed entry to confirm their feed is live const latestPosts = []; for (const author of authors) { const topic = Topic.fromString(author.topic); const owner = new EthAddress(author.owner); const reader = bee.makeFeedReader(topic, owner); try { const result = await reader.download(); console.log(`${author.name}: feed index ${result.feedIndex.toBigInt()}`); // Load the local post sidecar to get post data for the preview const postsFile = `${author.name.toLowerCase()}-posts.json`; const posts = JSON.parse(readFileSync(postsFile, "utf-8")); const latest = posts.at(-1); if (latest) { latestPosts.push({ author: author.name, ...latest }); } } catch { console.log(`${author.name}: no feed entries yet`); } } // Regenerate homepage with latest post previews from all authors const homeHTML = generateHomepageHTML(authors, latestPosts); const homeUpload = await bee.uploadFile(batchId, homeHTML, "index.html", { contentType: "text/html", }); const adminKey = new PrivateKey(cfg.admin.privateKey); const homeTopic = Topic.fromString(cfg.topics.home); const homeWriter = bee.makeFeedWriter(homeTopic, adminKey); await homeWriter.upload(batchId, homeUpload.reference); console.log("\nHomepage updated!"); console.log("View: " + `${process.env.BEE_URL}/bzz/${cfg.manifests.home}/`); function generateHomepageHTML(authors, latestPosts) { const cards = authors .map( (a) => { const latest = latestPosts.find((p) => p.author === a.name); const preview = latest ? `${latest.title} — ${latest.date}${latest.body.slice(0, 120)}…` : `No posts yet.`; return ` ${a.name} ${preview} `; } ) .join("\n"); return ` Multi-Author Blog Multi-Author Blog ${authors.length} author${authors.length !== 1 ? "s" : ""} ${cards} `; } ``` Run it after authors publish: ```bash node update-index.js # or: npm run update-index ``` The homepage now displays previews of the latest posts from all authors. The homepage feed manifest URL (`cfg.manifests.home`) always serves the aggregated view. :::info The `update-index.js` script reads local JSON sidecars (`alice-posts.json`, `bob-posts.json`) to populate post previews. In a production system, each post would be a separate Swarm upload, and the author's feed would store a JSON post-list reference (topic + manifest hash) instead of raw HTML. See [Etherjot](https://github.com/ethersphere/etherjot) for a full-featured example of this approach. ::: ### Read the Blog Any third party can read the blog and discover all authors using only the index feed manifest hash. No private keys are needed. Create `read.js`: ```js config(); const bee = new Bee(process.env.BEE_URL); const cfg = JSON.parse(readFileSync("config.json", "utf-8")); // Read the index feed to get the current authors manifest const indexTopic = Topic.fromString(cfg.topics.index); const indexOwner = new EthAddress(cfg.admin.owner); const indexReader = bee.makeFeedReader(indexTopic, indexOwner); const indexResult = await indexReader.downloadReference(); console.log("Index feed at index:", indexResult.feedIndex.toBigInt()); // Download the authors.json manifest const authorsData = await bee.downloadFile(indexResult.reference); const authors = JSON.parse(authorsData.data.toUtf8()); console.log(`\n${authors.length} authors in blog:\n`); // For each author, read their feed for (const author of authors) { const topic = Topic.fromString(author.topic); const owner = new EthAddress(author.owner); const reader = bee.makeFeedReader(topic, owner); try { const result = await reader.download(); console.log(`${author.name}`); console.log(` Feed index: ${result.feedIndex.toBigInt()}`); console.log(` URL: ${process.env.BEE_URL}/bzz/${author.feedManifest}/`); } catch { console.log(`${author.name}: feed not yet populated`); } } // Read the homepage feed const homeTopic = Topic.fromString(cfg.topics.home); const homeOwner = new EthAddress(cfg.admin.owner); const homeReader = bee.makeFeedReader(homeTopic, homeOwner); const homeResult = await homeReader.downloadReference(); console.log(`\nHomepage feed at index: ${homeResult.feedIndex.toBigInt()}`); console.log(`Homepage URL: ${process.env.BEE_URL}/bzz/${cfg.manifests.home}/`); ``` Run it: ```bash node read.js # or: npm run read ``` ### Adding a New Author Extending the system with a new author is straightforward. The new author gets their own key and topic. Their entry is appended to `authors.json`. Readers automatically discover them. The [`add-author.js`](https://github.com/ethersphere/examples/blob/main/multi-author-blog/add-author.js) script from the examples repo handles everything in one step. From the project directory (after running `init.js`): ```bash node add-author.js charlie # or: npm run add-author -- charlie ``` This will: 1. Generate a new private key for Charlie 2. Upload an initial empty blog page for them 3. Create their feed and feed manifest 4. Append their entry to `authors.json` and re-upload to the index feed 5. Update `config.json` so Charlie can use `add-post.js` Then run `update-index.js` to refresh the homepage with the new author. :::info Because the index feed always points to the *latest* `authors.json`, any reader who polls the index feed automatically discovers newly added authors. You don't need to notify readers through a separate channel — the feed is the notification channel. ::: ## Summary The multi-author blog demonstrates the key architectural pattern of large-scale Swarm applications: **composable feeds**. Key takeaways: - A feed entry can point to any Swarm content — HTML, JSON, images, or even the feed manifest hash of another feed. - Storing topic + owner + manifest hashes in a JSON document creates a directory of feeds — a linked feed network. - Feed manifest hashes are stable, permanent references. Use them as links between feeds. - Authors are independent. Each controls their own key and topic. Publishing a new post requires no coordination with other authors or the admin. - The homepage aggregator is a separate concern. It reads the index to discover authors, queries each author's feed for their latest content, and publishes the aggregated result. - Adding new authors does not break existing URLs. The index feed is updatable; readers poll it and automatically discover new entries. This architecture scales to hundreds of feeds and can represent complex data structures (threaded discussions, version hierarchies, category trees) — all composed from simple feed primitives and content-addressed storage. --- ## Developer Resources ## Learn ### Get Started - [What is Swarm](/docs/concepts/what-is-swarm) — Introduction to the decentralised storage network - [Developer introduction](/docs/develop/introduction) — Overview of building on Swarm - [Bee node quickstart](/docs/bee/installation/quick-start) — Get a node running in minutes - [Full installation guide](/docs/bee/installation/getting-started) — Step-by-step setup for all platforms - [Swarm papers ↗](https://papers.ethswarm.org) — Technical whitepapers and research - [The Book of Swarm ↗](https://docs.ethswarm.org/the-book-of-swarm.pdf) — Full technical book on Swarm's architecture and design - [Swarm Protocol Specification ↗](https://papers.ethswarm.org/p/swarm-protocol-spec/) — Formal spec for developers building clients or integrations ### Key Concepts - [DISC](/docs/concepts/DISC/) — How data is stored and retrieved across the network - [Incentives & BZZ](/docs/concepts/incentives/overview) — Economic model and token mechanics - [Postage stamps](/docs/develop/tools-and-features/buy-a-stamp-batch) — Pay for storage with stamp batches - [Feeds](/docs/develop/tools-and-features/feeds) — Mutable content pointers for updatable data - [Manifests](/docs/develop/tools-and-features/manifests) — Directory and routing structure on Swarm - [Chunk types](/docs/develop/tools-and-features/chunk-types) — Content-addressed and single-owner chunks - [Node types](/docs/bee/working-with-bee/node-types) — Full, light, and ultra-light participation levels ### Developer Guides - [Upload and download files](/docs/develop/upload-and-download) — Basic file operations via the API - [Host a website on Swarm](/docs/develop/host-your-website) — Deploy static sites to decentralised storage - [Work with files and directories](/docs/develop/files) — Manifest operations for virtual paths - [Routing manifests for SPAs](/docs/develop/routing) — Single-page app routing on Swarm - [Dynamic content with feeds](/docs/develop/dynamic-content) — Build updatable content with feed primitives - [Multi-author blog](/docs/develop/multi-author-blog) — Combine per-author feeds with a shared index - [Access control (ACT)](/docs/develop/act) — Restrict who can read your uploaded content - [PSS messaging](/docs/develop/tools-and-features/pss) — Send encrypted messages over the network - [GSOC messaging](/docs/develop/tools-and-features/gsoc) — Graffiti single-owner chunk messaging - [Encrypt uploads](/docs/develop/tools-and-features/store-with-encryption) — Client-side encryption before uploading - [Erasure coding](/docs/develop/tools-and-features/erasure-coding) — Redundant storage for fault tolerance - [Pinning content](/docs/develop/tools-and-features/pinning) — Keep content locally pinned to your node - [Run a local gateway proxy](/docs/develop/tools-and-features/gateway-proxy) — Serve Swarm content over HTTP - [Start a private test network](/docs/develop/tools-and-features/starting-a-test-network) — Local multi-node dev environment - [bee-factory](/docs/develop/tools-and-features/bee-dev-mode) — Local Swarm dev stack, no real funds needed ## Examples ### Publishing & Websites - [Cafe137/etherjot ↗](https://github.com/Cafe137/etherjot) — Static blog generator, live at etherjot.eth.limo - [ethersphere/examples ↗](https://github.com/ethersphere/examples) — Collection of example apps and starters - [examples/simple-blog ↗](https://github.com/ethersphere/examples/tree/main/simple-blog) — Minimal feed-backed blog, one publisher, updatable content - [examples/multi-author-blog ↗](https://github.com/ethersphere/examples/tree/main/multi-author-blog) — Multi-author blog using per-author feeds and a shared index - [examples/website ↗](https://github.com/ethersphere/examples/tree/main/website) — Upload a static website to Swarm, publish it to a feed for a stable URL, and resolve it via ENS - [examples/routing-manifest ↗](https://github.com/ethersphere/examples/tree/main/routing-manifest) — Manifest-based routing demo for single-page apps ### Streaming - [Solar-Punk-Ltd/swarm-stream-js ↗](https://github.com/Solar-Punk-Ltd/swarm-stream-js) — Core streaming library for Swarm - [Solar-Punk-Ltd/swarm-hls-stream ↗](https://github.com/Solar-Punk-Ltd/swarm-hls-stream) — HLS streaming over Swarm - [Solar-Punk-Ltd/swarm-stream-aggregator-js ↗](https://github.com/Solar-Punk-Ltd/swarm-stream-aggregator-js) — Stream aggregation layer - [Solar-Punk-Ltd/swarm-stream-react-example ↗](https://github.com/Solar-Punk-Ltd/swarm-stream-react-example) — React media player consuming Swarm streams - [Solar-Punk-Ltd/swarm-ingestion-stream-react-example ↗](https://github.com/Solar-Punk-Ltd/swarm-ingestion-stream-react-example) — React app for live stream ingest to Swarm ### Messaging & Chat - [Solar-Punk-Ltd/swarm-chat-js ↗](https://github.com/Solar-Punk-Ltd/swarm-chat-js) — Core chat library for Swarm - [Solar-Punk-Ltd/swarm-chat-aggregator-js ↗](https://github.com/Solar-Punk-Ltd/swarm-chat-aggregator-js) — Chat message aggregation layer - [Solar-Punk-Ltd/swarm-chat-react-example ↗](https://github.com/Solar-Punk-Ltd/swarm-chat-react-example) — React chat application on Swarm - [Solar-Punk-Ltd/swarm-comment-js ↗](https://github.com/Solar-Punk-Ltd/swarm-comment-js) — Core comments library - [Solar-Punk-Ltd/swarm-comment-react-example ↗](https://github.com/Solar-Punk-Ltd/swarm-comment-react-example) — React comments component - [Solar-Punk-Ltd/comment-system ↗](https://github.com/Solar-Punk-Ltd/comment-system) — Deployable comment system on Swarm - [Solar-Punk-Ltd/comment-system-ui ↗](https://github.com/Solar-Punk-Ltd/comment-system-ui) — UI for the Swarm comment system - [Solar-Punk-Ltd/swarm-collaborative-docs ↗](https://github.com/Solar-Punk-Ltd/swarm-collaborative-docs) — Real-time collaborative document editing on Swarm - [Cafe137/gsoc-group-chat ↗](https://github.com/Cafe137/gsoc-group-chat) — Group chat using GSOC (Graffiti Single Owner Chunks) ## Tools ### SDKs & APIs - [bee-js ↗](https://bee-js.ethswarm.org/docs/) — Official JavaScript / TypeScript SDK - [bee-js getting started ↗](https://bee-js.ethswarm.org/docs/getting-started/) — Quickstart for bee-js integration - [Bee HTTP API reference ↗](https://docs.ethswarm.org/api/) — Full REST API documentation - [swarm-cli](/docs/bee/working-with-bee/swarm-cli/) — Command-line interface for uploads, feeds, and more - [swarm-mcp ↗](https://github.com/ethersphere/swarm-mcp) — MCP server for AI agents to read and write Swarm - [Swarm Actions ↗](https://github.com/ethersphere/swarm-actions) — Deploy to Swarm from GitHub CI/CD workflows - [Swarm Quickstart Skills ↗](https://github.com/ethersphere/swarm-quickstart-skills) — Interactive Claude Code skills for guided Swarm onboarding ### Libraries & Primitives - [Solar-Punk-Ltd/file-manager-lib ↗](https://github.com/Solar-Punk-Ltd/file-manager-lib) — High-level file management primitives for Swarm - [Cafe137/feed-helper ↗](https://github.com/Cafe137/feed-helper) — Utility library for working with Swarm feeds - [fairDataSociety/fdp-storage ↗](https://github.com/fairDataSociety/fdp-storage) — Serverless web3 filesystem, Fair Data Protocol reference implementation ### Developer Tools - [ethersphere/create-swarm-app ↗](https://github.com/ethersphere/create-swarm-app) — Boilerplate for building Swarm apps with JavaScript - [Cafe137/fdp-play ↗](https://github.com/Cafe137/fdp-play) — Docker-based local Bee cluster and FDP dev environment - [Cafe137/pss-gsoc-learning-material ↗](https://github.com/Cafe137/pss-gsoc-learning-material) — Learning material for PSS and GSOC primitives - [examples/dynamic-content ↗](https://github.com/ethersphere/examples/tree/main/dynamic-content) — Content addressing and feeds, introduction example - [examples/filesystem ↗](https://github.com/ethersphere/examples/tree/main/filesystem) — Filesystem-style operations using bee-js - [agazso/swarm-cid-converter ↗](https://github.com/agazso/swarm-cid-converter) — Convert Swarm hashes or links to CID and vice versa - [Solar-Punk-Ltd/ipfs-to-swarm ↗](https://github.com/Solar-Punk-Ltd/ipfs-to-swarm) — Migrate data from IPFS to Swarm ### Gateways & Deploy - [Beeport ↗](https://beeport.ethswarm.org) — Upload without running a node - Public content gateway — Access any content by hash (`https://.bzz.link`) - ENS gateway — Resolve ENS names to Swarm content (`https://.eth.limo`) - [ENS name management ↗](https://app.ens.domains) — Register and manage ENS names - [ethersphere/gateway-proxy ↗](https://github.com/ethersphere/gateway-proxy) — Production gateway proxy server - [ethersphere/swarm-gateway ↗](https://github.com/ethersphere/swarm-gateway) — Official Swarm HTTP content gateway - [Swarmy ↗](https://swarmy.cloud) — Swarm as a service, upload and retrieve without running a node ### Network Tools - [SwarmScan ↗](https://swarmscan.io/) — Network explorer and node statistics - [Swarm Gateway ↗](https://gateway.ethswarm.org/) — Share files via URL - [Swarm Desktop ↗](https://ethswarm.org/build/desktop) — GUI node for non-technical users - [Bee Dashboard ↗](https://github.com/ethersphere/bee-dashboard) — Web UI for node management ## Community ### Community & Support - [Discord ↗](https://discord.ethswarm.org) — Community chat and support - [GitHub — Ethersphere org ↗](https://github.com/ethersphere) — All official repositories - [Awesome Swarm ↗](https://github.com/ethersphere/awesome-swarm) — Curated ecosystem list - [Blog ↗](https://blog.ethswarm.org) — News, updates, and deep dives - [Website ↗](https://ethswarm.org) — Main Swarm website - [FAQ](/docs/references/faq/) — Frequently asked questions - [SWIPs ↗](https://github.com/ethersphere/SWIPs) — Swarm Improvement Proposals, follow or contribute to protocol direction --- ## Website Routing In the [Manage Files](/docs/develop/files) guide you saw that a directory upload creates a manifest — a data structure mapping paths to content. Routing on Swarm is a direct consequence: every URL your site serves must correspond to an entry in that manifest. This guide covers the two main strategies for giving your site clean, navigable URLs. Swarm does not behave like a traditional web server — there is no server-side routing, and every route must correspond to a real file inside the site [manifest](./tools-and-features/manifests.md). If you try to use typical "clean URLs" like: ```bash /about /contact /dashboard/settings ``` Swarm will look for literal files such as: ```bash about contact dashboard/settings ``` ...which don’t exist if you've uploaded a static website with files like `about.html` and `contact.html`. There are two main strategies for addressing routing: * **Hash-Based Client-Side Routing** * **Manifest-Based Routing with Aliases or Index Files** Now let’s look at each method: ## Client-Side Hash Routing This section explains how to add hash based client side routing to your Swarm hosted site so that you can have clean URLs for each page of your website. See the [routing project in the examples repo](https://github.com/ethersphere/examples/tree/main/routing) for a full working example implementation. Swarm has no server backend running code and so can’t rewrite paths. One approach to routing is to set up a [SPA](https://en.wikipedia.org/wiki/Single-page_application) with React's `HashRouter`, which keeps all routing inside the browser. You can do this easily using a template from **create-swarm-app** and then adding your own pages. ### 1. Create a New Vite + React Project (with `create-swarm-app`) Run: ```bash npm init swarm-app@latest my-dapp vite-tsx ``` This generates a clean project containing: ```bash src/ App.tsx index.tsx config.ts public/ index.html package.json ``` You now have a fully working Vite/React app ready for Swarm uploads. ### 2. Install React Router Navigate to the project directory: ```bash cd my-dapp ``` Inside the project: ```bash npm install react-router-dom ``` This gives you client-side navigation capability. ### 3. Switch the App to Use Hash-Based Routing Swarm only serves literal files, so `/#/about` is the only reliable way to have “pages.” Replace your `./src/App.tsx` with: ```tsx export function App() { return ( } /> } /> } /> ) } ``` This gives you usable routes: ```bash /#/ → Home /#/about → About /#/anything → React 404 page ``` ### 4. Add Your Page Components Create your page components inside `./src`: Example `About.tsx`: ```tsx export function About() { return ( About This demo shows how to upload files or directories to Swarm using Bee-JS. ) } ``` Example `Home.tsx`: ```tsx export function Home() { return ( Home Welcome to your Swarm-powered app. ) } ``` Example `NotFound.tsx`: ```tsx export function NotFound() { return ( Page Not Found Return to Home ) } ``` ### 5. Add a Static `404.html` for Non-Hash URLs Swarm still needs a fallback for URLs like: ``` /non-existent-file ``` Create a `./public` directory and save a `404.html` file inside: ```html 404 – Not Found 404 This page doesn't exist. Return to Home ``` Vite will automatically include this in `dist/`. This file handles **non-hash** missing paths. React handles **hash** missing paths. ### 6. Build the Project Before uploading, compile the Vite app into a static bundle: ```bash npm run build ``` This produces a `dist/` folder containing: ```bash dist/ index.html 404.html assets/ ``` Everything inside `dist/` will be uploaded to your Swarm feed. ### 7. Deploy Site The project includes an `upload.js` script that uploads `./dist` to Swarm and publishes the result to a feed (so your URL stays stable across re-uploads). Set up your `.env` and run it: ```bash cp .env.example .env # Fill in BATCH_ID and PUBLISHER_KEY in .env npm run upload ``` You should see the website Swarm hash and a feed manifest hash. Open the feed manifest URL — `http://localhost:1633/bzz//` — in your browser. For details on feeds and stable URLs, see the [Host a Webpage](./host-your-website.md#advanced-keep-your-url-stable-across-updates) guide. The Home and About pages will be properly resolved by the routes we specified (`./#/`) and (`./#/about`), and non-existent URLs will be handled by the `NotFound` component for hash URLs and our `404.html` error document for all others. ![](/img/hash-routing.jpg) ## Manifest Based Routing The second routing method involves directly manipulating the manifest so that routes resolve properly to the intended content. ### 1. Upload the Site with Default Manifest Download example project: ```bash git clone https://github.com/ethersphere/examples.git cd examples/routing-manifest ``` Install dependencies: ```bash npm install ``` Configure environment variables. There is a `.env` file with important constants listed in it. You will need to at least update it with a valid batch ID, and potentially other changes if you are not using a default setup: Replace the value for `BATCH_ID` with your own valid batch IDs. Otherwise you may also need to update `BEE_URL` if its value doesn't match your Bee RPC endpoint. ```.env BEE_URL=http://localhost:1633 BATCH_ID=afd0810c2ea2936df849fe7b52650e231a19b7b31dbf7f96a93f0cf8a296f3f3 PUBLISHER_KEY=0x1111111111111111111111111111111111111111111111111111111111111111 UPLOAD_DIR=./site BASE_MANIFEST= ``` Start by uploading the site using the `upload.js` script provided in the example project: ```bash node .\upload.js ``` Terminal output: ```bash Manifest reference: e4d93dc161d9b1fb192fdcef7e18830bd50fa5b80561de18d5f2945fb8618515 URL: http://localhost:1633/bzz/e4d93dc161d9b1fb192fdcef7e18830bd50fa5b80561de18d5f2945fb8618515/ ``` Copy the manifest reference hash and set it inside your `.env` file as the BASE_MANIFEST value: ```bash BASE_MANIFEST=e4d93dc161d9b1fb192fdcef7e18830bd50fa5b80561de18d5f2945fb8618515 ``` If you peek inside the `upload.js` code you will see these lines: ```js const { reference } = await bee.uploadFilesFromDirectory( batchId, uploadDir, { indexDocument: "index.html", errorDocument: "404.html", } ) ``` Take note of the values set by the `indexDocument` and `errorDocument` options. With those options specified, manifest entries for the root path `./` and non existent paths will be created on upload to resolve to `index.html` and `404.html` respectively. Without specifying them, your site would not load either page unless you explicitly include the entire filename with extension in the URL. Navigate in your browser to the manifest URL output to the terminal after running the `upload.js` script: ```bash http://localhost:1633/bzz/e4d93dc161d9b1fb192fdcef7e18830bd50fa5b80561de18d5f2945fb8618515/ ``` Scroll down the example web page, read the instructions in the example website and click each of the example links to inspect how routing works by default without any manifest edits (besides those specified for the index and error documents): ![](/img/routing-manifest.png) You will find that links to direct files with file extensions included like `/about.html` will work, but links to `/about` will not. This is because we have not yet modified our manifest to set up typical routing behavior. ### 2. Fix Routing With Manifest Manipulation Without manifest edits, routes only work via exact file paths like: ``` /index.html /about.html /contact.html ``` Trying to access `/about` or `/about/` will fail. #### Add Routing Behavior by Manifest Manipulation We can easily add standard routing behavior by adding entries to our manifest which link the content reference with our desired URL paths: ```ts node.addFork('about', referenceForAbout, metadata) node.addFork('about/', referenceForAbout, metadata) ``` After this, both `/about` and `/about/` will resolve to the same content as `/about.html`. Run the provided script to update the manifest: :::warning Make sure you have already set the `BASE_MANIFEST` to the hash returned from the upload script we ran above or else this won't work: ```bash BASE_MANIFEST=e4d93dc161d9b1fb192fdcef7e18830bd50fa5b80561de18d5f2945fb8618515 ``` ::: ```bash node .\updateManifest.js ``` Terminal output: ```bash Updated manifest reference: 1483be29a42ec1e40b4d639f800a8fd982db9d5146088672a5edef9a1e0648aa URL: http://localhost:1633/bzz/1483be29a42ec1e40b4d639f800a8fd982db9d5146088672a5edef9a1e0648aa/007bff ``` Try navigating to the `/about` and `/about/` routes which previously led to a 404 page, they should now resolve properly. `/about.html` will still resolve properly as it did previously. ### 3. Remove Routes and Add New Ones Manifests control **which paths exist** on your site by mapping paths to immutable Swarm content references. To remove a page from your site, remove its route(s) from the manifest: ```js node.removeFork("old-page.html") ``` If you previously added clean-URL aliases such as: ```js node.addFork("about", referenceForAbout, metadata) node.addFork("about/", referenceForAbout, metadata) ``` …then you must remove **all** routes that expose that content: ```js node.removeFork("about") node.removeFork("about/") node.removeFork("about.html") ``` You can also reuse the **same content reference** under a different path by adding a new manifest entry that points to that same reference: ```js node.addFork("new-path", existingFileReference, metadata) ``` This is useful when you want the same page to be available at a new URL without re-uploading or changing the underlying content. If you remove the old route, the old URL will return a 404, while the new URL will serve the same content. Run the provided script to test the behavior: :::warning Make sure that once again you have updated the `.env` file with the hash returned from the second step before running this script. ::: ```bash node .\updateManifest.js ``` Terminal output: ```bash Updated manifest reference: 53e90f4033b99c7f6b82026b8f7beb39f42f99fbb816ae76e850cf2a1b45491d URL: http://localhost:1633/bzz/53e90f4033b99c7f6b82026b8f7beb39f42f99fbb816ae76e850cf2a1b45491d/ Routes now: /moved-about /moved-about/ Removed: /about /about/ /about.html ``` Now all our old links to the `about` page will 404. However, you can still reach the same content by navigating to the newly added routs: ```bash /moved-about /moved-about/ ``` Manually add them to the end of your website URL to check that they load properly. ### 4. Manifest Routing Enables Dynamic Content Once you understand manifest-based routing, you can dynamically: * Add new paths (e.g. blog posts, product pages) * Create custom routes * Remove unwanted paths --- **Next:** [Run a Gateway](/docs/develop/gateway-proxy) — expose your Swarm-hosted site to the public web through an HTTP gateway. --- ## AI Agent Skills [Swarm Quickstart Skills](https://github.com/ethersphere/swarm-quickstart-skills) is a set of interactive guides ("skills") that run inside [Claude Code](https://claude.com/product/claude-code). Instead of copying commands from the docs by hand, the skills check your prerequisites first, run real commands against your Bee node, and explain what is happening at each step. Type `/swarm` and you are routed to the right next step for your current setup — installing a node, buying a postage stamp, uploading files, or scaffolding a dApp. ## Requirements - [Claude Code](https://claude.com/product/claude-code) - Node.js 18+ - A running Bee light node at `http://localhost:1633` (for most skills) - The easiest way to install is to run `/swarm` in Claude Code, which installs and starts a light node for you. - Alternatively, check out the [quick start](./../../bee/installation/quick-start.md) ## Install Clone the repo and copy the `.claude/` folder into your project: ```bash git clone https://github.com/ethersphere/swarm-quickstart-skills.git cp -r swarm-quickstart-skills/.claude/ /path/to/your-project/ ``` Then open Claude Code in your project and start with the entry point: ```bash cd your-project && claude ``` Type `/swarm` to begin. See the [swarm-quickstart-skills repository](https://github.com/ethersphere/swarm-quickstart-skills) for the full list of available skills. For a programmatic alternative aimed at agents, see the [swarm-mcp](https://github.com/ethersphere/swarm-mcp) MCP server. --- ## Bee JS bee-js is Bee's complementary JavaScript library. It is the technology underpinning [swarm-cli](./../../bee/working-with-bee/swarm-cli.md) and [Swarm Desktop](./../../desktop/introduction.md) and is a powerful tool for building completely decentralized apps. See the [bee-js](https://bee-js.ethswarm.org/docs/) documentation for detailed information on using and installing the library. --- ## Postage Stamp Batches A postage batch is required to upload data to Swarm. Postage stamp batches represent _right to write_ data on Swarm's [DISC (Distributed Immutable Store of Chunks)](./../../concepts/DISC/DISC.mdx). The parameters which control the duration and quantity of data that can be stored by a postage batch are `depth` and `amount`, with `depth` determining data volume that can be uploaded by the batch and `amount` determining storage duration of data uploaded with the batch. :::info The storage volume and duration are both non-deterministic. Volume is non-deterministic due to the details of how [postage stamp batch utilization](./../../concepts/incentives/postage-stamps.md#batch-utilisation) works. While duration is non-deterministic due to price changes made by the [price oracle contract](./../../concepts/incentives/price-oracle.md). **Storage volume and `depth`:** When purchasing stamp batches for larger volumes of data (by increasing the `depth` value), the amount of data which can be stored becomes increasingly more predictable. For example, at `depth` 22 a batch can store between 2.28 GB (encrypted, paranoid erasure coding) and 17.18 GB (theoretical max), while at `depth` 28, a batch can store between 292.67 GB and 1.1 TB of data, and at higher depths the difference between the minimum and maximum storage volumes approach the same value. The effective volume also depends on the encryption and erasure coding settings used. See the [effective utilisation tables](./../../concepts/incentives/postage-stamps.md#effective-utilisation-tables) for the full details. **Storage duration and `amount`:** The duration of time for which a batch can store data is also non-deterministic since the price of storage is automatically adjusted over time by the [price oracle contract](./../../concepts/incentives/price-oracle.md). However, limits have been placed on how swiftly the price of storage can change, so there is no danger of a rapid change in price causing postage batches to unexpectedly expire due to a rapid increase in price. You can view a history of price changes by inspecting the events emitted by the oracle contract, or also through the [Swarmscan API](https://api.swarmscan.io/v1/events/storage-price-oracle/price-update). As you can see, if and when postage batch prices are updated, the updates are quite small. Still, since it is not entirely deterministic, it is important to monitor your stamp batch TTL (time to live) as it will change along with price oracle changes. You can inspect your batch's TTL using the `/stamps` endpoint of the API: ```bash root@noah-bee:~# curl -s localhost:1633/stamps | jq { "stamps": [ { "batchID": "f56af59cc2c785a3b45bbf3e46c3c4b20f80379339ef337b5bbf45ebe5629a66", "utilization": 0, "usable": true, "label": "", "depth": 17, "amount": "432072000", "bucketDepth": 16, "blockNumber": 38498819, "immutableFlag": true, "exists": true, "batchTTL": 82943 } ] } ``` Here we can see from the `batchTTL` that `82943` seconds remain, or approximately 23 hours. ::: For a deeper understanding of how `depth` and `amount` parameters determine the data volume and storage duration of a postage batch, see the [postage stamp page](./../../concepts/incentives/postage-stamps.md). ## Fund your node's wallet. In order to purchase a postage stamp batch, your node's Gnosis Chain address needs to be funded with sufficient xDAI to pay gas for transaction fees on Gnosis Chain as well as sufficient xBZZ to pay for the cost of the postage stamp batch itself. xBZZ can be obtained from a variety of different centralized and decentralized exchanges. You can find more information on [where to obtain xBZZ](https://www.ethswarm.org/get-bzz#how-to-get-bzz) on the Ethswarm homepage. xDAI can be obtained from a wide range of centralized and decentralized exchanges. See [this list of exchanges](https://docs.gnosischain.com/about/tokens/xdai) from the Gnosis Chain documentation to get started. You can learn more details from the [Fund Your Node](./../../bee/installation/fund-your-node.md) section. ## Buying a stamp batch When interacting with the Bee API directly, `amount` and `depth` are passed as path parameters: ```bash curl -s -X POST http://localhost:1633/stamps// ``` And with Swarm CLI, they are set using option flags: ```bash swarm-cli stamp buy --depth --amount ``` ```bash curl -s -X POST http://localhost:1633/stamps/100000000/20 ``` ```bash { "batchID": "8fcec40c65841e0c3c56679315a29a6495d32b9ed506f2757e03cdd778552c6b", "txHash": "0x51c77ac171efd930eca8f3a77e3fcd5aca0a7353b84d5562f8e9c13f5907b675" } ``` ```bash swarm-cli stamp buy --depth 20 --amount 100000000 ``` ```bash Estimated cost: 0.010 BZZ Estimated capacity: 4.00 GB Estimated TTL: 5 hours 47 minutes 13 seconds Type: Mutable When a mutable stamp reaches full capacity, it still permits new content uploads. However, this comes with the caveat of overwriting previously uploaded content associated with the same stamp. ? Confirm the purchase Yes Stamp ID: f4b9830676f4eeed4982c051934e64113dc348d7f5d2ab4398d371be0fbcdbf5 ``` :::info Once your batch has been purchased, it will take a few minutes for other Bee nodes in the Swarm to catch up and register your batch. Allow some time for your batch to propagate in the network before proceeding to the next step. ::: ## Setting stamp batch parameters and options When purchasing a batch of stamps there are several parameters and options which must be considered. The `depth` parameter will control how many chunks can be uploaded with a batch of stamps. The `amount` parameter determines how much xBZZ will be allocated per chunk, and therefore also controls how long the chunks will be stored. While the `immutable` header option sets the batch as either mutable or immutable, which can significantly alter the behavior of the batch utilisation (more details below). ### Choosing *depth* :::caution The minimum value for `depth` is 17, however a higher depth value is recommended for most use cases due to the [mechanics of stamp batch utilisation](./../../concepts/incentives/postage-stamps.md#batch-utilisation). See [the depths utilisation table](./../../concepts/incentives/postage-stamps.md#effective-utilisation-tables) to help decide which depth is best for your use case. ::: One notable aspect of batch utilisation is that the entire batch is considered fully utilised as soon as any one of its buckets are filled. This means that the actual amount of chunks storable by a batch is less than the nominal maximum amount. See the [postage stamp page](./../../concepts/incentives/postage-stamps.md) for a more complete explanation of how batch utilisation works and a [table](./../../concepts/incentives/postage-stamps.md#effective-utilisation-tables) with the specific amounts of data which can be safely uploaded for each `depth` value. ### Choosing *amount* :::caution The minimum `amount` value for purchasing stamps is required to be at least enough to pay for 24 hours of storage. To find this value multiply the lastPrice value from the postage stamp contract times 17280 (the number of blocks in 24 hours). You can also use the [calculator](#calculators) below. This requirement is in place in order to prevent spamming the network. ::: The `amount` parameter determines how much xBZZ is assigned per chunk for a postage stamp batch. You can use the calculators below to find the appropriate `amount` value for your target duration of storage and can also preview the price. For more information see the [postage stamp](./../../concepts/incentives/postage-stamps.md#batch-depth-and-batch-amount) page where a more complete description is included. ### Mutable or Immutable? Depending on the use case, uploaders may desire to use mutable or immutable batches. The fundamental difference between immutable and mutable batches is that immutable batches become unusable once their capacity is filled, while for mutable batches, once their capacity is filled, they may continue to be used, however older chunks of data will be overwritten with the newer once over capacity. The default batch type is immutable. In order to set the batch type to mutable, the `immutable` header should be set to `false`. See [this section on postage stamp batch utilisation](./../../concepts/incentives/postage-stamps.md#which-type-of-batch-to-use) to learn more about mutable vs immutable batches, and about which type may be right for your use case. ## Calculators The following postage batch calculators allow you to conveniently find the depth and amount values for a given storage duration and storage volume, or to find the storage duration and storage volume for a given depth and amount. The results will display the cost in xBZZ for the postage batch. The current pricing information is sourced from the Swarmscan API and will vary over time. :::info The 'effective volume' is the volume of data that can be safely stored for each batch depth, with a failure rate of less than 0.1%. The 'theoretical max volume' is significantly higher than the effective volume at lower depths, and the two values trend towards the same value at higher depths. Effective volumes are available for all depths from 17 to 41, and depend on the encryption and erasure coding settings selected. For example, at depth 17, the effective volume ranges from 13.17 kB (encrypted, paranoid) to 44.70 kB (unencrypted, no erasure coding). [Learn more here](./../../concepts/incentives/postage-stamps.md#effective-utilisation-tables). ::: ### Depth & Amount to Time & Volume Calculator ### Time & Volume to Depth & Amount Calculator The recommended depth in this calculator's results is the lowest depth value whose [effective volume](./../../concepts/incentives/postage-stamps.md#effective-utilisation-tables) is greater than the entered volume. ## Viewing Stamps To check on your stamps, send a GET request to the stamp endpoint. ```bash curl http://localhost:1633/stamps ``` ```bash { "stamps": [ { "batchID": "f4b9830676f4eeed4982c051934e64113dc348d7f5d2ab4398d371be0fbcdbf5", "utilization": 0, "usable": true, "label": "", "depth": 20, "amount": "100000000", "bucketDepth": 16, "blockNumber": 30643611, "immutableFlag": true, "exists": true, "batchTTL": 20588, "expired": false } ] } ``` ```bash swarm-cli stamp list ``` ```bash Stamp ID: f4b9830676f4eeed4982c051934e64113dc348d7f5d2ab4398d371be0fbcdbf5 Usage: 0% Remaining Capacity: 4.00 GB TTL: 5 hours 42 minutes 18 seconds Expires: 2023-10-26 ``` :::info It is not possible to reupload unencrypted content which was stamped using an expired postage stamp. ::: ## Checking the remaining TTL (time to live) of your batch :::info At present, TTL is a primitive calculation based on the current storage price and the assumption that storage price will remain static in the future. As more data is uploaded into Swarm, the price of storage will begin to increase. For data that it is important to keep alive, make sure your batches have plenty of time to live! ::: In order to make sure your *batch* has sufficient *remaining balance* to be stored and served by nodes in its [*area of responsibility*](./../../references/glossary.md#2-area-of-responsibility-related-depths), you must regularly check on its _time to live_ and act accordingly. The *time to live* is the number of seconds before the chunks will be considered for garbage collection by nodes in the network. The remaining *time to live* in seconds is shown in the API in the returned json object as the value for `batchTTL`, and with Swarm CLI you will see the formatted TTL as the `TTL` value. ```bash curl http://localhost:1633/stamps ``` ```bash { "stamps": [ { "batchID": "f4b9830676f4eeed4982c051934e64113dc348d7f5d2ab4398d371be0fbcdbf5", "utilization": 0, "usable": true, "label": "", "depth": 20, "amount": "100000000", "bucketDepth": 16, "blockNumber": 30643611, "immutableFlag": true, "exists": true, "batchTTL": 20588, "expired": false } ] } ``` ```bash swarm-cli stamp list ``` ```bash Stamp ID: f4b9830676f4eeed4982c051934e64113dc348d7f5d2ab4398d371be0fbcdbf5 Usage: 0% Remaining Capacity: 4.00 GB TTL: 5 hours 42 minutes 18 seconds Expires: 2023-10-26 ``` ## Top up your batch :::danger Don't let your batch run out! If it does, you will need to restamp and resync your content. ::: If your batch is starting to run out, or you would like to extend the life of your batch to protect against storage price rises, you can increase the batch TTL by topping up your batch using the stamps endpoint, passing in the relevant batchID into the HTTP PATCH request. ```bash curl -X PATCH "http://localhost:1633/stamps/topup/6d32e6f1b724f8658830e51f8f57aa6029f82ee7a30e4fc0c1bfe23ab5632b27/10000000" ``` List available stamps. ```bash swarm-cli stamp list ``` Copy stamp ID. ```bash Stamp ID: daa8c5b36e1cf481b10118a8b02430a6f22618deaa6ba5aa4ea660de66aa62db Usage: 13% Remaining Capacity: 3.50 GB TTL: 183 days 1 hour 37 minutes 8 seconds Expires: 2024-05-02 ``` Use `swarm-cli stamp topup` with the `--amount` and `--stamp` parameters set with the amount to topup in PLUR and the stamp ID. ```bash swarm-cli stamp topup --amount 10000000 --stamp daa8c5b36e1cf481b10118a8b02430a6f22618deaa6ba5aa 4ea660de66aa62db ``` Wait for topup transaction to complete. ```bash ⬡ ⬡ ⬢ Topup in progress. This may take a while. Stamp ID: daa8c5b36e1cf481b10118a8b02430a6f22618deaa6ba5aa4ea660de66aa62db Depth: 20 Amount: 100000001000 ``` ## Dilute your batch In order to store more data with a batch of stamps, you must "dilute" the batch. Dilution simply refers to increasing the depth of the batch, thereby allowing it to store a greater number of chunks. As dilution only increases the the depth of a batch and does not automatically top up the batch with more xBZZ, dilution will decrease the TTL of the batch. Therefore if you wish to store more with your batch but don't want to decrease its TTL, you will need to both dilute and top up your batch with more xBZZ. Here we call the `/stamps` endpoint and find a batch with `depth` 24 and a `batchTTL` of 2083223 which we wish to dilute: ```bash curl http://localhost:1633/stamps ``` ```json { "stamps": [ { "batchID": "0e4dd16cc435730a25ba662eb3da46e28d260c61c31713b6f4abf8f8c2548ae5", "utilization": 0, "usable": true, "label": "", "depth": 24, "amount": "10000000000", "bucketDepth": 16, "blockNumber": 29717348, "immutableFlag": false, "exists": true, "batchTTL": 2083223, "expired": false } ] } ``` Next we call the [`dilute`](/api/#tag/Postage-Stamps/paths/~1stamps~1dilute~1{batch_id}~1{depth}/patch) endpoint to increase the `depth` of the batch using the `batchID` and our new `depth` of 26: ```bash curl -s -XPATCH http://localhost:1633/stamps/dilute/0e4dd16cc435730a25ba662eb3da46e28d260c61c31713b6f4abf8f8c2548ae5/26 ``` And a `txHash` of our successful transaction: ```bash { "batchID": "0e4dd16cc435730a25ba662eb3da46e28d260c61c31713b6f4abf8f8c2548ae5", "txHash": "0x298e80358b3257292752edb2535a1cd84440c074451b61f78fab349aea4962b7" } ``` And finally we use the `/stamps` endpoint again to confirm the new `depth` and decreased `batchTTL`: ```bash curl http://localhost:1633/stamps ``` We can see the new `depth` of 26 and a decreased `batchTTL` of 519265. ```json { "stamps": [ { "batchID": "0e4dd16cc435730a25ba662eb3da46e28d260c61c31713b6f4abf8f8c2548ae5", "utilization": 0, "usable": true, "label": "", "depth": 26, "amount": "10000000000", "bucketDepth": 16, "blockNumber": 29717348, "immutableFlag": false, "exists": true, "batchTTL": 519265, "expired": false } ] } ``` List available stamps, make sure to use the `--verbose` flag so that we can see the batch depth. ```bash swarm-cli stamp list --verbose ``` We have a stamp batch with depth 20 we want to dilute. Copy stamp ID of that batch. ```bash Listing postage stamps... Stamp ID: daa8c5b36e1cf481b10118a8b02430a6f22618deaa6ba5aa4ea660de66aa62db Usage: 13% Remaining Capacity: 3.50 GB Total Capacity (mutable): 4.00 GB TTL: 182 days 4 hours 39 minutes 47 seconds Expires: 2024-05-02 Depth: 20 Bucket Depth: 16 Amount: 100010002000 Usable: true Utilization: 2 Block Number: 29734329 ``` Use `swarm-cli stamp dilute` with the `--depth` and `--stamp` parameters set with the desired new depth and the stamp ID. ```bash swarm-cli stamp dilute --depth 21 --stamp daa8c5b36e1cf481b10118a8b02430a6f22618deaa6ba5aa4ea660de66aa62db ``` ```bash ⬡ ⬡ ⬢ Dilute in progress. This may take a while. Stamp ID: daa8c5b36e1cf481b10118a8b02430a6f22618deaa6ba5aa4ea660de66aa62db Depth: 20 Amount: 100010002000 ``` ## Stewardship The stewardship endpoint in combination with [pinning](./pinning.md) can be used to guarantee that important content is always available. It is used for checking whether the content for a Swarm reference is retrievable and for re-uploading the content if it is not. An HTTP GET request to the `stewardship` endpoint checks to see whether the content for the specified Swarm reference is retrievable: :::info `stewardship` is not currently supported by Swarm CLI ::: ```bash curl "http://localhost:1633/stewardship/c0c2b70b01db8cdfaf114cde176a1e30972b556c7e72d5403bea32e c0207136f" ``` ```json { "isRetrievable": true } ``` If the content is not retrievable, an HTTP PUT request can be used to re-upload the content: ```bash curl -X PUT "http://localhost:1633/stewardship/c0c2b70b01db8cdfaf114cde176a1e30972b556c7e72d5403bea32ec0207136f" ``` Note that for the re-upload to succeed, the associated content must be available locally, either pinned or cached. Since it isn't easy to predict if the content will be cached, for important content pinning is recommended. --- ## Swarm Cheatsheet The Swarm Cheatsheet is a dense, two-page quick-reference for building on Swarm. It's designed to print cleanly to A4, so you can keep it beside you at a hackathon or on your desk — use the **Download PDF** link below to grab a copy. Download PDF --- ## Chunk Types Swarm is home to many types of chunks, but these can be categoried into 4 broad categories. Read [The Book of Swarm](https://www.ethswarm.org/the-book-of-swarm-2.pdf) for more information on how swarm comes together. ## Content Addressed Chunks Content addressed chunks are chunks whose addresses are determined by the BMT hashing algorithm. This means you can be sure that all content addressed chunks content is already verified - no more need to check md5 hashes of your downloaded data! :::warning To be able trust your data, you must run your own Bee node that automatically verifies data, using gateways puts your trust in the gateway operators. ::: ## Trojan Chunks Trojan chunks are a special version of content addressed chunks that have been 'mined' so that their natural home is in a particular area of the Swarm. If the destination node is in the right neighborhood, it will be able to receive and decrypt the message. See [PSS](./pss.md) for more information, or check out the [bee-js](https://bee-js.ethswarm.org/docs/api/classes/Bee/#psssend) bindings. ## Single Owner Chunks Single Owner Chunks are distinct from Trojan and Content Addressed Chunks and are the only other type of chunk which is allowed in Swarm. These chunks represent part of Swarm's address space which is reserved just for your personal Ethereum key pair! Here you can write whatever you'd please. Single Owner Chunks are the technology that powers Swarm's [feeds](./feeds.md), but they are capable of much more! Look out for more chats about this soon, and for more info read [The Book of Swarm](https://www.ethswarm.org/the-book-of-swarm-2.pdf). ## Custom Chunk Types Although all chunks must satisfy the constraints of either being addressed by the BMT hash of their payload, or assigned by the owner of an Ethereum private key pair, so much more is possible. How else can you use the DISC to distribute and store your data? We're excited to see what you come up with! 💡 Share your creations in the [#builders](https://discord.gg/8SMCfvm3kw) channel of our [Discord Server](https://discord.gg/kHRyMNpw7t). --- ## bee-factory `bee-factory` is the recommended way to run a local Swarm development environment. It spins up 5 Bee nodes connected to a local Anvil blockchain — all wired together in a single command, with no real xBZZ required. :::info The `bee dev` command is no longer available. Please use `bee-factory` for local development instead. ::: To test against a real public network instead of a local stack, you can also run a node against the [Sepolia testnet](../../bee/working-with-bee/configuration.md#sepolia-testnet-configuration). ## Requirements - Node.js ≥ 18 - Docker ## Installation ```sh npm install -g @ethersphere/bee-factory ``` ## Usage ```sh bee-factory start # Start the stack (uses bundled snapshot for fast boot) bee-factory start --fresh # Redeploy contracts from scratch, save new snapshot bee-factory start --tag v2.7.1 # Build Bee from a specific git ref (default: master) bee-factory stop # Stop and remove all containers ``` The `--fresh` flag redeploys all contracts and saves a new snapshot; subsequent normal starts load from it instantly. ## Endpoints Once running, the nodes are accessible at these addresses: | Node | API | P2P | |----------|-------------------------|-------------------------| | Queen | http://localhost:1633 | http://localhost:1634 | | Worker 1 | http://localhost:11633 | http://localhost:11634 | | Worker 2 | http://localhost:21633 | http://localhost:21634 | | Worker 3 | http://localhost:31633 | http://localhost:31634 | | Worker 4 | http://localhost:41633 | http://localhost:41634 | **Anvil RPC:** `http://localhost:8545` (chain ID 1337) ## Deployed contracts The following contracts are deployed automatically on startup, with addresses printed to the console: | Contract | Role | |----------------------|---------------------------| | BzzToken | ERC-20 BZZ token | | PostageStamp | Postage stamp management | | PriceOracle | Postage pricing | | StakeRegistry | Node staking | | Redistribution | Stake redistribution | | SimpleSwapFactory | Swap contract factory | | SwapPriceOracle | Swap pricing oracle | ## Notes - Each node is funded with 1 ETH and 100 BZZ. - Node password: `bee-factory` - Uses [Foundry test keys](https://www.getfoundry.sh/anvil#default-accounts) — never use in production. --- ## Erasure Coding(Tools-and-features) [Erasure coding](./../../concepts/DISC/erasure-coding.md) is a powerful method for safeguarding data, offering robust protection against partial data loss. This technique involves dividing the original data into multiple fragments and generating extra parity fragments to introduce redundancy. A key advantage of erasure coding is its ability to recover the complete original data even if some fragments are lost. Additionally, it offers the flexibility to customize the level of data loss protection, making it a versatile and reliable choice for preserving data integrity on Swarm. For a more in depth dive into erasure coding on Swarm, see the [erasure coding paper](https://papers.ethswarm.org/p/erasure/) from the Swarm research team. ## Uploading With Erasure Coding Erasure coding is available for the [`/bytes`](/api/#tag/Bytes) and [`/bzz`](/api/#tag/BZZ) endpoints, however it is not available for the [`/chunks`](/api/#tag/Chunk) endpoint which deals with single chunks. Since erasure coding relies on splitting data into chunks and the chunk is the smallest unit of data within Swarm which cannot be further subdivided, erasure coding is not applicable for the `/chunks` endpoint which deals with single chunks. To upload data to Swarm using erasure coding, the `swarm-redundancy-level: ` header is used: ```bash curl \ -X POST http://localhost:1633/bzz?name=test.txt \ -H "swarm-redundancy-level: 1" \ -H "swarm-postage-batch-id: 54ba8e39a4f74ccfc7f903121e4d5d0fc40732b19efef5c8894d1f03bdd0f4c5" \ -H "Content-Type: text/plain" \ --data-binary @test.txt {"reference":"c02e7d943fbc0e753540f377853b7181227a83e773870847765143681511c97d"} ``` The accepted values for the `swarm-redundancy-level` header range from the default of 0 up to 4. Each level corresponds to a different level of data protection, with erasure coding turned off at 0, and at its maximum at 4. Each increasing level provides increasing amount of data redundancy offering greater protection against data loss. | Redundancy Level Value | Level Name | | ---------------- | --------- | | 1 | Medium | | 2 | Strong | | 3 | Insane | | 4 | Paranoid | For more details about each level of protection refer to the [erasure coding page](./../../concepts/DISC/erasure-coding.md) in the learn section and refer to the [erasure coding paper](https://papers.ethswarm.org/p/erasure/) for an even deeper dive. ## Cost Calculator Widget This calculator takes as input an amount of data and an erasure coding redundancy level, and outputs the number of additional parity chunks required to erasure code that amount of data as well as the increase in cost to upload vs. a non-erasure encoded upload: For more details of erasure coding costs, see [here](./../../concepts/DISC/erasure-coding.md). ## Downloading Erasure Encoded Data For a downloader, the process for downloading a file which has been erasure encoded does not require any changes from the [normal download process](./../upload-and-download.md). There are several options for adjusting the default behaviour for erasure encoded downloads, however there is no need to adjust them. ### Default Download Behaviour Erasure coding retrieval for downloads is enabled by default, so there is no need for a downloader to explicitly enable the feature. The default download behaviour is to use the DATA strategy with fallback enabled. With these settings, first an attempt will be made to download the data chunks only. If any of the data chunks are missing, then the retrieval method will fall back to the RACE strategy (PROX is not currently implemented and so will be skipped). With the RACE strategy, an attempt will be made to download all data and parity chunks, and chunks will continue to be downloaded until enough have been retrieved to reconstruct the original data. ### Options :::warning Do not adjust these options unless you know exactly what you are doing. The default settings are the best option for almost all cases. ::: When downloading erasure encoded data, there are three related headers which may be used: `swarm-redundancy-strategy`, `swarm-redundancy-fallback-mode: `, and `swarm-chunk-retrieval-timeout`. * `swarm-redundancy-strategy`: This header allows you to set the retrieval strategy for fetching chunks. The accepted values range from 0 to 3. Each number corresponds to a different chunk retrieval strategy. The numbers stand for the NONE, DATA, PROX and RACE strategies respectively which are described in greater detail in [the API reference](/api/#tag/BZZ) (also see [the erasure code paper](https://papers.ethswarm.org/p/erasure/) for even more in-depth descriptions). With each increasing level, there will be a potentially greater bandwidth cost. :::info Retrieval Strategies 0. NONE: This strategy is based on direct retrieval of data chunks without pre-fetching, with parity chunks ignored. No pre-fetching is used (data chunks are fetched sequentially). 1. DATA: The same as NONE, except that data chunks are pre-fetched (data chunks are fetched in parallel in order to reduce latency). 2. PROX: For this strategy, the chunks closest (in Kademlia distance) to the node are retrieved first. *(Not yet implemented.)* 3. RACE: Initiates requests for all data and parity chunks and continues to retrieve chunks until enough chunks are retrieved that the original data can be reconstructed. ::: * `swarm-redundancy-fallback-mode: `: Enables the fallback feature for the redundancy strategies so that if one of the retrieval strategies fails, it will fallback to the more intensive strategy until retrieval is successful or retrieval fails. Default is `true`. * `swarm-chunk-retrieval-timeout: `: Allows you to specify the timeout time for chunk retrieval with a default value of 30 seconds. *(This is primarily used by the Bee development team for testing and it's recommended that Bee users do not need to use this option.)* An example download request may look something like this: ```bash curl -OJL \ -H "swarm-redundancy-strategy: 3" \ -H "swarm-redundancy-fallback-mode: true" \ http://localhost:1633/bzz/c02e7d943fbc0e753540f377853b7181227a83e773870847765143681511c97d/ % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 ``` For this request, the redundancy strategy is set to 3 (RACE), which means that it will initiate a request for all data and parity chunks and continue to retrieve chunks until enough have been retrieved to reconstruct the source data. This is in contrast with the default strategy of DATA where only the data chunks will be retrieved. However, it is recommended to not adjust the default settings for these options, so a typical request would actually look like this (which is the exact same as a [normal download](./../upload-and-download.md) without any additional options set): ```bash curl -OJL http://localhost:1633/bzz/c02e7d943fbc0e753540f377853b7181227a83e773870847765143681511c97d/ % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 ``` This means that there is no need to inform downloaders that a file uses erasure coding, as even with the default download behaviour reconstruction of the source file will be attempted if any chunks are missing. --- ## Feeds Swarm feeds cleverly combine [single owner chunks](./chunk-types.md) into a data structure which enables you to have static addresses for your mutable content. This means that you can signpost your data for other Bees, and then update it at will. :::info Although it's possible to interact with feeds directly, it can involve a little data juggling and crypto magic. For the easiest route, see [the bee-js feeds functionality](./bee-js.md) and [swarm-cli](./../../bee/working-with-bee/swarm-cli.md), or for the super 1337, share your implementations in other languages in the [#builders](https://discord.gg/8SMCfvm3kw) channel of our [Discord Server](https://discord.gg/kHRyMNpw7t). ::: ## What are Feeds? A feed is a collection of Single Owner Chunks with predicatable addresses. This enables creators to upload pointers to data so that consumers of the feed are able to find the data in Swarm using only an _Ethereum address_ and _Topic ID_. ## Creating and Updating a Feed In order to edit a feed, you will need to sign your chunks using an Ethereum keypair. For the intrepid, check out the [The Book of Swarm](https://www.ethswarm.org/the-book-of-swarm-2.pdf) on precise details on how to do this. For the rest of us, both [bee-js](./bee-js.md) and [swarm-cli](./../../bee/working-with-bee/swarm-cli.md) provide facilities to achieve this using JavaScript and a node-js powered command line tool respectively. ## No More ENS Transaction Charges Swarm's feeds provide the ability to update your immutable content in a mutable world. Simply reference your feed's `manifest address` as the `content hash` in your ENS domain's resolver, and Bee will automatically provide the latest version of your website. ## Use Cases for Feeds Feeds are a hugely versatile data structure. They allow you to host frequently updated content such as websites, RSS feeds (for podcasts, news, etc.), or even a DNS style architecture on top of Swarm's decentralized DISC. --- ## Gateway Proxy The [Swarm Gateway](https://github.com/ethersphere/swarm-gateway) is the standard way to expose a Bee node over HTTP. :::info Another tool which is currently popular for running Bee in gateway mode is [Gateway Proxy](https://github.com/ethersphere/gateway-proxy). It offers several features not yet included in Swarm Gateway. However, since it is set for deprecation, unless you have a specific need, it is recommended to use Swarm Gateway instead. ::: It acts as a reverse proxy that runs in front of a Bee node, allowing you to expose your node publicly. It proxies the Bee HTTP API and content endpoints, while optionally adding access control, postage batch auto-buy, and other optional features. ## Public Access to Swarm A gateway can be used to run a public endpoint that allows users to: * Access content stored on Swarm using standard HTTP URLs * Browse websites hosted on Swarm * Interact with Swarm through a familiar web interface This makes Swarm content accessible to any web client, even if the user is not running a Bee node locally. ## Authentication, Access Control, and Policy The Swarm Gateway also acts as an access control and content moderation layer in front of a Bee node. Rather than exposing a Bee node directly to the public internet, the gateway allows operators to place a managed HTTP interface in front of it. Through this interface, the gateway can: * Expose a Bee node through a single public HTTP endpoint * Restrict or control uploads and other sensitive operations * Require authentication for selected endpoints or request types * Apply basic access control and usage policies before requests reach the Bee node This makes it possible to run public, private, or semi-public gateways while retaining control over how the underlying Bee node is used. For production deployments, the gateway is typically run behind an HTTPS reverse proxy to ensure encrypted connections. ## Stamp Management The Swarm Gateway can optionally manage postage stamps on behalf of the operator, including: * Automatically buying new batches * Monitoring batch usage and expiration * Keeping batches alive based on specified TTL This is especially useful for gateways that accept uploads from users or applications. ## Setting up a Gateway For a step by step guide on setting up a gateway yourself, refer to the [guide in the Develop on Swarm section](./../gateway.md). --- ## GSOC ## Introduction The Graffiti Several Owner Chunk (GSOC) feature enables a single Bee *service* node to receive messages from multiple Bee *writer* nodes. It is based on a [Single Owner Chunk (SOC)](./chunk-types.md#single-owner-chunks) with an address which is derived so that it falls within the neighborhood of the service node, ensuring updates are automatically synced as part of the normal full node syncing process. The service node determines the data used to derive the GSOC private key. Any node with access to this data can derive the same private key and update the GSOC in order to send messages to the service node. Since only full nodes sync neighborhood chunks, the service node *must be a full node to receive GSOC updates*. To receive messages in real time, the service node establishes a WebSocket connection to listen for GSOC update events. When a matching SOC update reaches a node with an active GSOC connection, an event is emitted, enabling the service node to dynamically receive messages as part of a many-to-one notification system. :::info GSOC was initially introduced in a [SWIP](https://github.com/ethersphere/SWIPs/blob/99e6cf90a4768b24d27e5339b205c18825b53322/SWIPs/swip-draft_graffiti-soc.md#gsoc-identifier), which outlines its core concepts and implementation details, and it is an evolution of the earlier [Graffiti feed](https://github.com/fairDataSociety/FIPs/blob/master/text/0062-graffiti-feed.md) feature. ::: ## *bee-js* GSOC Methods While you can interact with GSOC directly via the `/gsoc/subscribe/{address}` endpoint, the [bee-js](./bee-js.md) library is the recommended way for most users. The library includes three methods which make it easy to get started with GSOC: ### `Bee.gsocMine()` The `Bee.gsocMine` method mines a GSOC private key corresponding to a specific overlay address: - The service node uses this method to generate the private key for a GSOC in its own neighborhood, and then uses it with the `gsocSubscribe` method to listen for updates from writer nodes. - A writer node uses this method to generate the private key it uses to send messages to the service node with the `gsocSend()` method. #### Parameters - **`targetOverlay`** (`PeerAddress | Uint8Array | string`) – The overlay address of the service node. - **`identifier`** (`Identifier | Uint8Array | string`) – A unique, arbitrary value that can be modified to mine a GSOC private key derived from a specific value. - **`proximity`** (`number`, default: `16`) – Determines the neighborhood depth, i.e., how many prefix bits match between `targetOverlay` and the mined GSOC overlay address. The function returns a mined private key, which corresponds to a GSOC overlay address that falls within the `targetOverlay` neighborhood. #### Functionality: 1. Mines and returns a private key that generates a GSOC overlay address within the specified `proximity` of `targetOverlay`. 2. The service node uses this method to mine a GSOC chunk whose overlay falls within its own neighborhood, and shares the values used as input with the writer node (`targetOverlay`, `identifier`, and `proximity`). 3. The writer node uses this method with the input values shared from the service node to generate the private key that allows it to send messages as updates to the mined GSOC. This function allows users to derive a GSOC overlay address that aligns with a target node’s network neighborhood. ### `Bee.gsocSend()` The `Bee.gsocSend` method is used by a writer node for sending GSOC messages. It creates an update for the GSOC using the provided `data` as the message, signs the update with the private key mined by the `gsocMine()` method, and uploads it to Swarm. #### Parameters: - **`postageBatchId`** (`BatchId | Uint8Array | string`) – The ID of the postage batch used to pay for the upload. - **`signer`** (`PrivateKey | Uint8Array | string`) – The private key used to sign the chunk. - **`identifier`** (`Identifier | Uint8Array | string`) – A unique identifier for the GSOC. - **`data`** (`string | Uint8Array`) – The payload to be sent. - **`options`** (`UploadOptions`, optional) – Additional upload configuration. - **`requestOptions`** (`BeeRequestOptions`, optional) – Custom request options. #### Functionality: 1. Used by the writer node to send a GSOC message using the private key returned from `gsocMine()`. 2. Requires the `postageBatchId` for a valid postage stamp batch (ideally [mutable](./gsoc.md#script-requirements)) to send messages. ### `Bee.gsocSubscribe()` The `Bee.gsocSubscribe` method is used by the service node to establish a WebSocket connection to listen for GSOC messages. It subscribes to messages associated with a specific `address` and `identifier`. #### Parameters: - **`address`** (`EthAddress | Uint8Array | string`) – The Gnosis Chain address associated with the private key returned by the `gsocMine()` function. - **`identifier`** (`Identifier | Uint8Array | string`) – A unique identifier used to track the messages. - **`handler`** (`GsocMessageHandler`) – A callback function to handle incoming messages. #### Functionality: 1. The function is used by the service node to construct a GSOC address using the provided `identifier` and `address`. 2. A WebSocket connection is opened to subscribe to update events for this GSOC address. 3. Incoming messages are processed by the `handler` function. 4. The function returns a `GsocSubscription` object with a `cancel` method to terminate the subscription. ## Example Scripts The service node and writer node scripts below are a minimalistic example of how to use `bee-js` to set up a service node to listen for GSOC messages, and a writer node to send GSOC messages. ### Script Requirements To run both nodes and send messages from the writer node to the service node you will need: 1. A fully synced Bee full node for the service node and a second Bee light node for the writer node (they do not both need to be running on the same machine) 2. A small amount of xDAI (~0.01) and xBZZ (~0.01) 3. [NodeJS](https://nodejs.org/en) & [NPM](https://www.npmjs.com/) 4. A mutable stamp batch (*set the* [`immutable` header parameter](/api/#tag/Postage-Stamps/paths/~1stamps~1%7Bamount%7D~1%7Bdepth%7D/post) *to `false` when* [buying a batch](./buy-a-stamp-batch.md#buying-a-stamp-batch)) :::warning Only ***mutable*** postage stamp batches should be used for GSOC. Since each GSOC update utilizes one slot within the ***same*** [postage batch bucket](./../../concepts/incentives/postage-stamps.md#batch-utilisation), immutable batches will fill up very quickly (e.g., at depth 18, four GSOC messages exhaust the batch). Mutable batches allow updates to overwrite older ones, preventing full utilization and enabling indefinite GSOC messaging as long as the batch still has remaining TTL. ::: ### Service Node Script ✅ For your service node project, you must use a ***full node***. ❌ A service node does not need a postage stamp batch. #### Initialize Project First, initialize the service node project on a machine running a full Bee node in the background: ```bash mkdir service-node cd service-node npm init -y && npm pkg set type="module" && cat package.json npm install bee-js --save ``` The command first creates the `service-node` directory, moves into that directory, initializes a `package.json` file, sets `"type": "module"` in the file, and finally installs the `bee-js` library. Next create a file named `index.js` which will hold the code for our service node. ```bash touch index.js ``` Then open in your editor of choice: ```bash vi index.js ``` Copy the completed code below for a service node into our newly created `index.js` file: :::tip Read through the code and code comments for a more in-depth understanding of how the service node works. ::: ```javascript // Configuration const BEE_HOST = 'http://localhost:1633'; // Change this if necessary const BEE_PROXIMITY = 12 // Mining depth of the GSOC overlay - modified from the default of 16 for a shorter mining time const BEE = new Bee(BEE_HOST, {}); async function mineGsocKey() { console.log('Fetching node addresses...'); const addresses = await BEE.getNodeAddresses(); const privateKey = BEE.gsocMine(addresses.overlay, NULL_IDENTIFIER, BEE_PROXIMITY); // `NULL_IDENTIFIER` is a constant `Uint8Array(32)` imported from `bee-js` for use as a default identifier console.log('Mining completed. Public Key:', privateKey.publicKey().toCompressedHex()); return privateKey; } async function createGsocListener() { try { const privateKey = await mineGsocKey(); // Subscribe to GSOC messages const subscription = BEE.gsocSubscribe(privateKey.publicKey().address(), NULL_IDENTIFIER, { onMessage: message => console.log('Received GSOC update:', message.toJSON()), onError: err => console.error('Error in subscription:', err), }); console.log('Listening for GSOC updates...'); return { privateKey, subscription }; } catch (err) { console.error('Error:', err.message); } } (async () => { await createGsocListener(); })(); ``` #### Update Configuration Update the constants in the configuration section with your own information: * Update `BEE_HOST` if your node is not using the default `http://localhost:1633`. #### Run Service Node Script Start the service node: ```bash node index.js ``` If everything is working correctly, after a few seconds you should see output like this: ```bash Fetching node addresses... Node overlay address: 75703155f54cbb899a359a7e3daec75da7722baef9286522e58e86ccbfcd7f13 Mining completed. Public Key: e82d2c98a92a3b0c690f6ba28070c59e3e0cd0a2a384d3b03cba9d1fded41a9831e73a3232d85b3614833d344c7d502dd09d7ecd0614b06095c86be0c8501460 Listening for GSOC updates... ``` This means the service node has successfully mined a GSOC chunk that it falls into its own neighborhood, and is now listening for updates on that chunk. Copy the `Node overlay address:` value (`75703155f54cbb899a359a7e3daec75da7722baef9286522e58e86ccbfcd7f13` from the example output) and save it - we will need it for our writer node's configuration. ### Writer Node Script ✅ For your writer node, either a light or a full node can be used ✅ A writer node needs a valid ***mutable*** (not technically required, but [strongly recommended](./gsoc.md#script-requirements)) postage stamp batch in order to send GSOC messages #### Initialize Project We initialize our writer node using almost the same command as our service node, only the directory name has been changed. ```bash mkdir writer-node cd writer-node npm init -y && npm pkg set type="module" && cat package.json npm install bee-js --save ``` The command first creates the `writer-node` directory, moves into that directory, initializes a `package.json` file, sets `"type": "module"` in the file, and finally installs the `bee-js` library. Next create a file named `index.js` which will hold the code for our writer node. ```bash touch index.js ``` Then open in your editor of choice: ```bash vi index.js ``` Copy the completed code below for a writer node into our newly created `index.js` file: :::tip Read through the code and code comments for a more in-depth understanding of how the writer node works. ::: ```javascript // Configuration const BEE_HOST = 'http://localhost:1643'; // Change this if necessary const BEE_BATCH = '42a10176596ecc73dcd24b91a16fb77d874ebd108fe8bc7fb896c8e89e8cb06e'; // Ensure this is a valid hex string const TARGET_OVERLAY = '75703155f54cbb899a359a7e3daec75da7722baef9286522e58e86ccbfcd7f13'; // Overlay of service node writer node wants to message const BEE_PROXIMITY = 12 // Mining depth of the GSOC overlay - modified from the default of 16 for a shorter mining time const BEE = new Bee(BEE_HOST, {}); async function mineGsocKey() { const privateKey = BEE.gsocMine(TARGET_OVERLAY, NULL_IDENTIFIER, BEE_PROXIMITY); // `NULL_IDENTIFIER` is a constant `Uint8Array(32)` imported from `bee-js` for use as a default identifier console.log('Mining completed. Public Key:', privateKey.publicKey().toCompressedHex()); return privateKey; } async function sendGsocMessage(privateKey, name, body) { if (!privateKey) { console.error('Error: Private key is not available'); return; } if (!/^[0-9a-fA-F]{64}$/.test(BEE_BATCH)) { console.error('Error: Invalid BEE_BATCH. It must be a 64-character hex string.'); return; } const message = JSON.stringify({ name, body }); await BEE.gsocSend(BEE_BATCH, privateKey, NULL_IDENTIFIER, message); console.log('Message sent:', message); } (async () => { const privateKey = await mineGsocKey(); // Example: Sending a message after a delay (simulate user input) setTimeout(() => { sendGsocMessage(privateKey, 'Alice', 'Hello from Node.js!'); }, 5000); })(); ``` #### Update Configuration Update the configuration section constants with your own information: * Set `BEE_HOST` to your writer node's API endpoint * Set `BEE_BATCH` to the batch id of a valid, *mutable* postage stamp batch - [buy a batch](./buy-a-stamp-batch.md) if needed * Set `TARGET_OVERLAY` to the service node overlay value we copied from the output of the service node script After updating the configuration, run the writer node script (before running the writer node script, make sure the service node script has already been started and is currently listening for GSOC updates): #### Run Writer Node Script ```bash node index.js ``` If everything is working correctly, after a few moments on your writer node you should see output like this: ```bash Mining completed. Public Key: e82d2c98a92a3b0c690f6ba28070c59e3e0cd0a2a384d3b03cba9d1fded41a9831e73a3232d85b3614833d344c7d502dd09d7ecd0614b06095c86be0c8501460 Message sent: {"name":"Alice","body":"Hello from Node.js!"} ``` While in the output from our service node, we should receive the update: ```bash Received GSOC update: { name: 'Alice', body: 'Hello from Node.js!' } ``` Congratulations! You've just sent your first GSOC message. --- ## Hosting Your Dapps & Storing Their Data Swarm is hugely versatile, but at a very basic level you can think of it as storage for your dapps data that is too big for blockchain, but still needs to live in our totally decentralised universe. Swarm is perfect for storing your NFT meta-data and images in a web3 way that won't break the bank and can live forever! ## Tools and Features Swarm is designed with decentralised applications in mind, and much time has been devoted to designing tools and features to support their prototyping and development. ### AI Agent Skills In a hurry? The [Swarm Quickstart Skills](./ai-agent-skills.md) run inside Claude Code and walk you through setting up a Bee node and building on Swarm — just type `/swarm` and follow the guided steps. ### Bee JS Our maverick JavaScript team, the Bee-Gees (🕺), have been working hard in the last few months to build some impressive tools for all you budding dapp developer Bees to get stuck into! Find out how to use the [bee-js](./bee-js.md) JavaScript library to start creating your own that live and work on Swarm! ### Chunk Types Swarm contains 3 types of chunks which enable us to build novel structures of how data can be stored in the swarm - in a completely decentralised way. Learn more about [chunk types](./chunk-types.md) to change the way you deal with data in your dapps forever! ### Feeds Swarm's single owner chunks have been cleverly combined to create user generated [feeds](./feeds.md) in the swarm, see this example of how chunks are combined into a useful data structure you can use to build amazing applications. ### PSS Hey there! Pss! 🤫 Swarm's trojan chunks are implemented in Bee to deliver [Postal Service on Swarm](./pss.md) - a pub-sub system that provides a totally leak-proof messaging system over the swarm. ### Gateway Proxy If you want your users to be able to access Swarm without running their own Bee node, for the time being you will need to make use of the [Gateway Proxy tool](https://github.com/ethersphere/gateway-proxy). Join us in the [#builders](https://discord.gg/8SMCfvm3kw) room in our [Discord Server](https://discord.gg/kHRyMNpw7t) for more information on how to make your Swarm based applications accessible to everyone. ### Local Development with bee-factory If you want to test Swarm-based applications without spending real xBZZ, [bee-factory](./dev-mode.md) is the recommended tool. It starts a full local stack — 5 Bee nodes plus a local Anvil blockchain — with a single command. The `bee dev` mode was removed in Bee v2.8.1; use bee-factory instead. ### Starting a Test Network While bee-factory already runs multiple nodes locally, setting up a [test network](./starting-a-test-network.md) gives you even greater control over simulating interactions between nodes in a more customised environment. --- ## Manifests Manifests define how files and folders are organized in Swarm. Instead of a flat list of uploaded files, Bee encodes directory structure as a compact prefix [trie](https://en.wikipedia.org/wiki/Trie). This allows URLs like `/images/logo.png`, `/docs/readme.txt`, or `/` to resolve efficiently to the correct Swarm references. Whenever you upload a directory — via `/bzz`, `bee-js`, or `swarm-cli` — Bee automatically creates and uploads a manifest that enables a filesystem-like layer inside Swarm. The manifest reference itself is the root reference for your uploaded directory. Manifests provide: * Filesystem-style path lookup (`/foo/bar.txt`) * Hierarchical directory structure * Metadata attached to files or folders (e.g., Content-Type) * Optional custom routing behavior for websites (via manifest configuration) They allow structured collections of files — including websites — to exist naturally on Swarm. ## Why Manifests Matter Raw content hashes identify data immutably, but they don’t express relationships between files. A manifest adds this missing structure: it groups related files, assigns paths, stores metadata, and exposes the entire folder tree through URL-like navigation. Without manifests, every application on Swarm would need its own indexing and routing logic. ## When Manifests Are Created Manifests are created whenever you upload a directory via the `/bzz` endpoint, which is used internally by `swarm-cli` and `bee-js` for directory uploads. Bee scans the folder, builds the trie, and produces a manifest reference representing the entire directory tree. By contrast: * `/bytes` and `/chunks` upload raw binary data only * They do not create manifests ## Index and Error Document Options Directory uploads in `bee-js` support two optional helpers: ```js { indexDocument: "index.html", errorDocument: "404.html" } ``` These specify which file Bee should serve for the manifest root (`/`) and for invalid paths. These options can be used with normal directory uploads, not only websites — and any file type can be used — not only HTML. ## How Manifests Are Structured A manifest is structured as a trie: nodes are connected by forks, and each fork is labelled with a path segment. The path to a node is the concatenation of the segments you follow from the root. If a node’s `target` is non-zero, the path represented by that node refers to a file and the target points to its Swarm content. If the `target` is zero, the node behaves like a directory or intermediate prefix. The printed output below shows a decoded Mantaray manifest (using the [`manifestToJson.js` script](https://github.com/ethersphere/examples/blob/main/utils/manifestToJson.js) from the examples repo). It represents a simple folder tree containing a root file and a nested subfolder. :::info About the Term "Mantaray" "Mantaray" was originally a standalone Swarm library for working with manifests. It has since been integrated into `bee-js` and is no longer maintained as a standalone library. Its name is still used in `bee-js` for the manifest-related classes (`MantarayNode`, etc.). ::: ```json { "path": "/", "target": "0x0000000000000000000000000000000000000000000000000000000000000000", "metadata": null, "forks": { "folder/": { "path": "folder/", "target": "0x0000000000000000000000000000000000000000000000000000000000000000", "metadata": null, "forks": { "nested.txt": { "path": "nested.txt", "target": "0x9442e445c0d58adea58e0a8afcdcc28ed7642d7a4ff9a253e8f1595faafbb808", "metadata": { "Content-Type": "text/plain; charset=utf-8", "Filename": "nested.txt" }, "forks": {} }, "subfolder/deep.txt": { "path": "subfolder/deep.txt", "target": "0x6aa935879ad2a547e57ea6350338bd04ad758977b542e86b31c159f31834b8fc", "metadata": { "Content-Type": "text/plain; charset=utf-8", "Filename": "deep.txt" }, "forks": {} } } }, "root.txt": { "path": "root.txt", "target": "0x98e63f7e826a01634881874246fc873cdf06bb5409ff5f9ec61d1e2de1dd3bf6", "metadata": { "Content-Type": "text/plain; charset=utf-8", "Filename": "root.txt" }, "forks": {} } } } ``` ### Key Concepts **Node** — Represents either a directory or a file inside the manifest. - Directories have a zero target and may contain child nodes. - Files have a non-zero target pointing to their Swarm content. **Fork** — A mapping from a path segment to a child node. In the JSON representation, the fork is the key (for example `"folder/"` or `"root.txt"`), and the value is the child node for that segment. **Path** — The path segment label stored on a node (the same string used as the fork key from its parent). It may be a single segment such as `folder/` or `nested.txt`, or a remainder of the full path such as `subfolder/deep.txt`. **Target** — The Swarm reference for a file’s content. Directories use a zero target. **Metadata** — Attributes stored with a node (for example `Content-Type`, filename, etc.). ## Immutability Manifests are immutable. When you add, remove, or move a file, Bee writes new manifest nodes rather than modifying existing ones. Each update produces a new manifest reference, and older versions remain accessible. To provide a stable entry point even as the manifest changes, you can combine manifests with feeds. A feed acts as an updateable pointer: publish each new manifest reference to the feed, and users access the feed hash instead of individual manifest hashes. You can find examples of this in the [Building on Swarm](./../introduction.md) page. ## Serving Files From a Manifest A manifest reference acts like the root of a filesystem. Requests such as: ``` / → index document /docs/readme.txt → file content ``` are resolved by walking the trie until the correct file target is found. Bee handles this automatically under: ``` /bzz// ``` Paths to directories such as `/docs/` or even `/` will result in a 404 error by default unless the manifest is modified (or by specifying an `indexDocument` for `/`): ``` /docs/ → 404 ``` You can specify which file or webpage you would like paths such as `/docs/` (which do not get entries in the manifest by default) to resolve to by manipulating the manifest. See the ["Filesystem"](./../files.md) and [Routing](./../routing.md) guides for more information and examples. :::caution The `target` values inside a manifest should not be accessed directly. They cannot be reliably fetched via endpoints such as `/bzz/` or tools like `swarm-cli download`. To retrieve a file, always access it through the manifest, for example: ``` curl http://localhost:1633/bzz//root.txt -o ./root.txt ``` or ``` swarm-cli download c8275d246e8a14ccd6f680ea0ecae543ebc0734e52676a5468a9a30db156be64/disc.jpg ``` Bee resolves the underlying content automatically and returns the file correctly. ::: ## When to Modify the Manifest Most Swarm users never need to manually inspect or modify a manifest. When you upload a directory, Bee creates one automatically and it "just works" for many common cases. You only need to modify the manifest when you want to change how paths resolve after the upload. ### Websites For simple single-page sites, no manual changes are required — setting `indexDocument` and `errorDocument` during upload is enough. You need to modify the manifest when you want to change routing behavior, such as: * Removing `.html` extensions for clean URLs * Adding, changing, or deleting routes * Redirecting paths or restructuring the site See the [Routing](./../routing.md) guide for details. ### Directory Uploads For one-time directory uploads that you don’t plan to change, you typically don’t need to touch the manifest. However, if you later want to add files, remove files, rename paths, or point new paths at existing content, the manifest must be updated. See the ["Filesystem"](./../files.md) guide for examples. ## Putting It All Together A manifest turns a set of immutable chunks into a structured, navigable collection of files. It enables folder trees, static assets, multi-file application bundles, websites, and data archives to exist on Swarm in a coherent, accessible way. Whether you're uploading a small directory or a full site, the manifest is what ties everything together. --- ## Pinning Each Bee node is configured to reserve a certain amount of memory on your computer's hard drive to store and serve chunks within their _neighborhood of responsibility_ for other nodes in the Swarm network. Once this alloted space has been filled, each Bee node deletes older chunks to make way for newer ones as they are uploaded by the network. Each time a chunk is accessed, it is moved back to the end of the deletion queue, so that regularly accessed content stays alive in the network and is not deleted by a node's garbage collection routine. Bee nodes provide a facility to **pin** important content so that it is not deleted by the node's garbage collection routine. Chunks can be _pinned_ either during upload, or retrospectively using the Swarm reference. ## How do I pin content during upload? {#pin-during-upload} To store content so that it will persist even when Bee's garbage collection routine is deleting old chunks, we simply pass the `Swarm-Pin` header set to `true` when uploading. ```bash curl -H "Swarm-Pin: true" -H "Swarm-Postage-Batch-Id: 78a26be9b42317fe6f0cbea3e47cbd0cf34f533db4e9c91cf92be40eb2968264" --data-binary @bee.mp4 localhost:1633/bzz\?bee.mp4 ``` ```json { "reference": "1bfe7c3ce4100ae7f02b62e38d3e8d4c3a86ea368349614a87827402f20cbb30" } ``` ## How do I manage pinned content? {#administer-pinned-content} To check what content is currently pinned on your node, query the `pins` endpoint of your Bee API: ```bash curl localhost:1633/pins ``` ```json { "references": [ "1bfe7c3ce4100ae7f02b62e38d3e8d4c3a86ea368349614a87827402f20cbb30" ] } ``` or, to check for specific references: ```bash curl localhost:1633/pins/1bfe7c3ce4100ae7f02b62e38d3e8d4c3a86ea368349614a87827402f20cbb30 ``` A `404` response indicates the content is not available. ### How do I unpin content? {#unpinning-content} We can unpin content by sending a `DELETE` request to the pinning endpoint using the same reference: ````bash curl -XDELETE http://localhost:1633/pins/1bfe7c3ce4100ae7f02b62e38d3e8d4c3a86ea368349614a87827402f20cbb30 `` ```json {"message":"OK","code":200} ```` Now, when check again, we will get a `404` error as the content is no longer pinned. ```bash curl localhost:1633/pins/1bfe7c3ce4100ae7f02b62e38d3e8d4c3a86ea368349614a87827402f20cbb30 ``` ```json { "message": "Not Found", "code": 404 } ``` :::info Pinning and unpinning is possible for files (as in the example) and also the chunks, directories, and bytes endpoints. See the [API](/api/) documentation for more details. ::: ### How do I pin already-uploaded content? {#pinning-already-uploaded-content} The previous example showed how we can pin content upon upload. It is also possible to pin content that is already uploaded and present in the Swarm. To do so, we can send a `POST` request including the swarm reference to the files pinning endpoint. ```bash curl -X POST http://localhost:1633/pins/7b344ea68c699b0eca8bb4cfb3a77eb24f5e4e8ab50d38165e0fb48368350e8f ``` ```json { "message": "OK", "code": 200 } ``` The `pins` operation will attempt to fetch the content from the network if it is not available on the local node. Now, if we query our files pinning endpoint again, the swarm reference will be returned. ```bash curl http://localhost:1633/pins/7b344ea68c699b0eca8bb4cfb3a77eb24f5e4e8ab50d38165e0fb48368350e8f ``` ```json { "reference": "7b344ea68c699b0eca8bb4cfb3a77eb24f5e4e8ab50d38165e0fb48368350e8f" } ``` :::warning While the pin operation will attempt to fetch content from the network if it is not available locally, we advise you to ensure that the content is available locally before calling the pin operation. If the content, for whatever reason, is only fetched partially from the network, the pin operation only partly succeeds and leaves the internal administration of pinning in an inconsistent state. ::: --- ## PSS Messaging Out of the ashes of Ethereum's vision for a leak-proof decentralised anonymous messaging system - Whisper - comes PSS (or BZZ, whispered! 🤫). Swarm provides the ability to send messages that appear to be normal Swarm traffic, but are in fact messages that may be received and decrypted to reveal their content only by the specific nodes they were intended to be received by. PSS provides a pub-sub facility that can be used for a variety of tasks. Nodes are able to listen to messages received for a specific topic in their nearest neighborhood and create messages destined for another neighborhood which are sent over the network using Swarm's usual data dissemination protocols. ## Prerequisites :::warning **You must be running a full node to receive PSS messages.** Ultra-light and light nodes cannot subscribe to or receive messages. Full nodes connected to a blockchain RPC endpoint are required for PSS functionality. ::: Additionally, to send PSS messages, you will need: - A postage stamp batch with sufficient xBZZ balance - The recipient's Swarm address prefix (at least 2 bytes) - The recipient's public key ### Subscribe and Receive Messages Once your Bee node is up and running, you will be able to subscribe to feeds using WebSockets. For testing, it is useful to use the [websocat](https://docs.rs/crate/websocat/1.0.1) command line utility. Here we subscribe to the topic `test-topic` ```bash websocat ws://localhost:1633/pss/subscribe/test-topic ``` Our node is now watching for new messages received in its nearest neighborhood. :::info Because a message is disguised as a normal chunk in Swarm, you will receive the message upon syncing the chunk, even if your node is not online at the moment when the message was send to you. ::: ### Send Messages Messages can be sent simply by sending a `POST` request to the PSS API endpoint. When sending messages, we must specify a 'target' prefix of the recipient's Swarm address, a partial address representing their neighborhood. Currently the length of this prefix is recommended to be two bytes, which will work well until the network has grown to a size of ca. 20-50K nodes. We must also provide the public key, so that Bee can encrypt the message in such a way that it may only be read by the intended recipient. For example, if we want to send a PSS message with **topic** `test-topic` to a node with address... `7bc50a5d79cb69fa5a0df519c6cc7b420034faaa61c175b88fc4c683f7c79d96` ...and public key... `0349f7b9a6fa41b3a123c64706a072014d27f56accd9a0e92b06fe8516e470d8dd` ...we must include the **target** `7bc5` and the public key itself as a query argument. ```bash curl -H "Swarm-Postage-Batch-Id: 78a26be9b42317fe6f0cbea3e47cbd0cf34f533db4e9c91cf92be40eb2968264" -X POST \ localhost:1833/pss/send/test-topic/7bc5?recipient=0349f7b9a6fa41b3a123c64706a072014d27f56accd9a0e92b06fe8516e470d8dd \ --data "Hello Swarm" ``` More information on how to buy a postage stamp batch and get its batch id can be found [here](./buy-a-stamp-batch.md). ### Send Messages in a Test Network Now, let's see this in action by setting up two Bee nodes on a test network, connecting them, and sending PSS messages from one to the other. First start two Bee nodes. We will start them with distinct ports for the API and p2p ports, since they will be running on the same computer. Run the following command to start the first node. Note that we are passing `""` to the `--bootnode` argument so that our nodes will not connect to a network. ```bash bee start \ --api-addr=:1833 \ --data-dir=/tmp/bee2 \ --bootnode="" \ --p2p-addr=:1834 \ --blockchain-rpc-endpoint=http://localhost:8545 ``` We must make a note of the Swarm overlay address, underlay address and public key which are created once each node has started. We find this information from the `/addresses` endpoint of the API. ```bash curl -s localhost:1833/addresses | jq ``` ```json { "overlay": "46275b02b644a81c8776e2459531be2b2f34a94d47947feb03bc1e209678176c", "underlay": [ "/ip4/127.0.0.1/tcp/7072/p2p/16Uiu2HAmTbaZndBa43PdBHEekjQQEdHqcyPgPc3oQwLoB2hRf1jq", "/ip4/192.168.0.10/tcp/7072/p2p/16Uiu2HAmTbaZndBa43PdBHEekjQQEdHqcyPgPc3oQwLoB2hRf1jq", "/ip6/::1/tcp/7072/p2p/16Uiu2HAmTbaZndBa43PdBHEekjQQEdHqcyPgPc3oQwLoB2hRf1jq" ], "ethereum": "0x0b546f2817d0d889bd70e244c1227f331f2edf74", "public_key": "03660e8dbcf3fda791e8e2e50bce658a96d766e68eb6caa00ce2bb87c1937f02a5" } ``` Now the same for the second node. ```bash bee start \ --api-addr=:1933 \ --data-dir=/tmp/bee3 \ --bootnode="" \ --p2p-addr=:1934 \ --blockchain-rpc-endpoint=http://localhost:8545 ``` ```bash curl -s localhost:1935/addresses | jq ``` ```json { "overlay": "085b5cf15a08f59b9d64e8ce3722a95b2c150bb6a2cef4ac8b612ee8b7872253", "underlay": [ "/ip4/127.0.0.1/tcp/7073/p2p/16Uiu2HAm5RwRgkZWxDMAff2io6L4Qd1uL9yNgZSNTCdPsukcg5Qr", "/ip4/192.168.0.10/tcp/7073/p2p/16Uiu2HAm5RwRgkZWxDMAff2io6L4Qd1uL9yNgZSNTCdPsukcg5Qr", "/ip6/::1/tcp/7073/p2p/16Uiu2HAm5RwRgkZWxDMAff2io6L4Qd1uL9yNgZSNTCdPsukcg5Qr" ], "ethereum": "0x9ec47bd86a82276fba57f3009c2f6b3ace4286bf", "public_key": "0289634662d3ed7c9fb1d7d2a3690b69b4075cf138b683380023d2edc2e6847826" } ``` Because we configured the nodes to start with no bootnodes, neither node should have peers yet. ```bash curl -s localhost:1833/peers | jq ``` ```bash curl -s localhost:1935/peers | jq ``` ```json { "peers": [] } ``` Let's connect node 2 to node 1 using the localhost (127.0.0.1) underlay address for node 1 that we have noted earlier. ```bash curl -X POST \ localhost:1935/connect/ip4/127.0.0.1/tcp/1834/p2p/16Uiu2HAmP9i7VoEcaGtHiyB6v7HieoiB9v7GFVZcL2VkSRnFwCHr ``` Now, if we check our peers endpoint for node 1, we can see our nodes are now peered together. ```bash curl -s localhost:1833/peers | jq ``` ```json { "peers": [ { "address": "a231764383d7c46c60a6571905e72021a90d506ef8db06750f8a708d93fe706e" } ] } ``` Of course, since we are p2p, node 2 will show node 1 as a peer too. ```bash curl -s localhost:1935/peers | jq ``` ```json { "peers": [ { "address": "7bc50a5d79cb69fa5a0df519c6cc7b420034faaa61c175b88fc4c683f7c79d96" } ] } ``` We will use `websocat` to listen for the PSS messages' Topic ID `test-topic` on our first node. ```bash websocat ws://localhost:1833/pss/subscribe/test-topic ``` Now we can use PSS to send a message from our second node to our first node. Since our first node has a 2 byte address prefix of `a231`, we will specify this as the `targets` section in our POST request's URL. We must also include the public key of the recipient as a query parameter so that the message can be encrypted in a way only our recipient can decrypt. ```bash curl \ -H "Swarm-Postage-Batch-Id: 78a26be9b42317fe6f0cbea3e47cbd0cf34f533db4e9c91cf92be40eb2968264" -X POST "localhost:1933/pss/send/test-topic/7bc5?recipient=0349f7b9a6fa41b3a123c64706a072014d27f56accd9a0e92b06fe8516e470d8dd" \ --data "Hello Swarm" ``` The PSS API endpoint will now create a PSS message for its recipient in the form of a 'Trojan Chunk' and send this into the network so that it may be pushed to the correct neighborhood. Once it is received by its recipient it will be decrypted and determined to be a message with the topic we are listening for. Our second node will decrypt the data and we'll see a message pop up in our `websocat` console! ```bash websocat ws://localhost:1833/pss/subscribe/test-topic ``` ``` Hello Swarm ``` Congratulations! 🎉 You have sent your first encrypted, zero leak message over Swarm! --- ## Starting a Private Network A private network can be used to test your applications in an isolated environment before you deploy to Swarm mainnet. It can be started by overriding the default configuration values of your Swarm node. Throughout this tutorial, we will make use of configuration files to configure the nodes but of course you can also do the same using flags or environment variables (see [Start your node](./../../bee/working-with-bee/configuration.md)). ## Start a network on your own computer ### Configuration Starting a network is easiest achieved by making use of configuration files. We need at least two nodes to start a network. Hence, below two configuration files are provided. Save them respectively as `config_1.yaml` and `config_2.yaml`. **config_1.yaml** ```yaml network-id: 7357 api-addr: 127.0.0.1:1633 p2p-addr: :1634 bootnode: "" data-dir: /tmp/bee/node1 password: set-a-strong-password swap-enable: false mainnet: false blockchain-rpc-endpoint: https://sepolia.dev.fairdatasociety.org verbosity: 5 full-node: true ``` **config_2.yaml** ```yaml network-id: 7357 api-addr: 127.0.0.1::1733 p2p-addr: :1734 data-dir: /tmp/bee/node2 bootnode: "" password: set-a-strong-password welcome-message: "Bzz Bzz Bzz" swap-enable: false mainnet: false blockchain-rpc-endpoint: https://sepolia.dev.fairdatasociety.org verbosity: 5 full-node: true ``` Note that for each node, we provide a different `api-addr`. If we had not specified different addresses here, we would get an `address already in use` error, as no two applications can listen to the same port. We also specify a different `p2p-addr`. If we had not, our nodes would not be able to communicate with each other. We also specify a separate `data-dir` for each node, as each node must have its own separate key and chunk data store. We also provide a network-id, so that our network remains separate from the Swarm mainnet, which has network-id 1. Nodes will not connect to peers which have a different network id. We also set our bootnode to be the empty string `""`. A bootnode is responsible for bootstrapping the network so that a new node can find its first few peers before it begins its own journey to find friends in the Swarm. In Swarm any node can be used as a bootnode. Later, we will use our first node as the bootnode for our other node(s), but for now we leave this option blank. We have set `mainnet` to false so that our node runs on the Sepolia testnet, and we provide an RPC endpoint for Sepolia in the `blockchain-rpc-endpoint` option. We have also set `full-node` and `swap-enable` to `true` so that we can run full nodes. Log verbosity has been set to level 5 with the `verbosity` option. By setting it at the highest level of 5, we make sure all important information is shown in our logs. Setting this is optional. Finally, note the `welcome-message` in the first nodes configuration file. This is a friendly feature allowing you to send a message to peers that connect to you! ### Starting Your Nodes Now we have created our configuration files, let's start our nodes by running `bee start --config config_1.yaml`, then in another Terminal session, run `bee start --config config_2.yaml`. We can now inspect the state of our network by sending HTTP requests to the [API](/api/). ```bash curl -s http://localhost:1633/topology | jq .connected ``` ``` 0 ``` ```bash curl -s http://localhost:1733/topology | jq .connected ``` ``` 0 ``` No connections yet? Right! Let's remedy that! :::info Here we are using the `jq` command line utility to count the amount of objects in the `peers` array in the JSON response we have received from our API, learn more about how to install and use `jq` [here](https://jqlang.org/). ::: ### Making a network In order to create a network from our two isolated nodes, we must first instruct our nodes to connect to each other. This step is not explicitly needed if you connect to the main Swarm network, as the default bootnodes in the Swarm network will automatically suggest peers. First, we will need to find out the network address of the first node. To do this, we send a HTTP request to the `addresses` endpoint of the API. ```bash curl localhost:1633/addresses | jq ``` ```json { "overlay": "b1978be389998e8c8596ef3c3a54214e2d4db764898ec17ec1ad5f19cdf7cc59", "underlay": [ "/ip4/127.0.0.1/tcp/1634/p2p/QmQHgcpizgoybDtrQXCWRSGdTP526ufeMFn1PyeGd1zMEZ", "/ip4/172.25.128.69/tcp/1634/p2p/QmQHgcpizgoybDtrQXCWRSGdTP526ufeMFn1PyeGd1zMEZ", "/ip6/::1/tcp/1634/p2p/QmQHgcpizgoybDtrQXCWRSGdTP526ufeMFn1PyeGd1zMEZ" ], "ethereum": "0xd22cc790e2aef341827e1e49cc631d2a16898cd9", "publicKey": "023b26ce8b78ed8cdb07f3af3d284c95bee5e038e7c5d0c397b8a5e33424f5d790", "pssPublicKey": "039ceb9c1f0afedf79991d86d89ccf4e96511cf656b43971dc3e878173f7462487" } ``` Here, we get firstly the **overlay address** - this is the permanent address Swarm uses as your anonymous identity in the network and secondly, a list of all the [multiaddresses](https://libp2p.io/docs/peers/#peer-ids-in-multiaddrs), which are physical network addresses at which you node can be found by peers. Note the addresses starting with an `/ip4`, followed by `127.0.0.1`, which is the `localhost` internal network in your computer. Now we can use this full address to be the bootnode of our second node so that when it starts up, it goes to this address and both nodes become peers of each other. Let's add this into our config_2.yaml file. **config_2.yaml** ```yaml network-id: 7357 api-addr: 127.0.0.1::1733 p2p-addr: :1734 data-dir: /tmp/bee/node2 bootnode: "/ip4/127.0.0.1/tcp/1634/p2p/QmQHgcpizgoybDtrQXCWRSGdTP526ufeMFn1PyeGd1zMEZ" password: set-a-strong-password welcome-message: "Bzz Bzz Bzz" swap-enable: false blockchain-rpc-endpoint: https://sepolia.dev.fairdatasociety.org verbosity: 5 full-node: true ``` Now, we can shut our second node and reboot with the new configuration. Look at the the output for your first node, you should see our connection message! Let's also verify that we can see both nodes in using each other's API's. ```bash curl -s http://localhost:1633/peers | jq ``` ```bash curl -s http://localhost:1733/peers | jq ``` Congratulations! You have made your own tiny two bee Swarm! 🐝 🐝 ## Funding Nodes While you have successfully set up two nodes, they are currently unfunded with either sETH or sBZZ. Sepolia ETH (sETH) is required for issuing transactions on the Sepolia testnet, and Sepolia BZZ (sBZZ) is required for your node to operate as a full staking node. To fund our nodes, we need to first collect the blockchain addresses for each node. We can use the `/addresses` endpoint for this: ```bash curl localhost:1633/addresses | jq ``` ```bash { "overlay": "b1978be389998e8c8596ef3c3a54214e2d4db764898ec17ec1ad5f19cdf7cc59", "underlay": [ "/ip4/127.0.0.1/tcp/1634/p2p/QmQHgcpizgoybDtrQXCWRSGdTP526ufeMFn1PyeGd1zMEZ", "/ip4/172.25.128.69/tcp/1634/p2p/QmQHgcpizgoybDtrQXCWRSGdTP526ufeMFn1PyeGd1zMEZ", "/ip6/::1/tcp/1634/p2p/QmQHgcpizgoybDtrQXCWRSGdTP526ufeMFn1PyeGd1zMEZ" ], "ethereum": "0xd22cc790e2aef341827e1e49cc631d2a16898cd9", "publicKey": "023b26ce8b78ed8cdb07f3af3d284c95bee5e038e7c5d0c397b8a5e33424f5d790", "pssPublicKey": "039ceb9c1f0afedf79991d86d89ccf4e96511cf656b43971dc3e878173f7462487" } ``` Then copy the address in the "ethereum" field. This is the address you need to send sETH and sBZZ to. You will need to send only a very small amount of sETH such as 0.01 sETH, to get started. You will need 10 sBZZ to run a full node with staking. After sending sETH and sBZZ to your node's address which you copied above, restart your node and it should begin operating properly as a full node. Repeat these same steps with the other node in order to complete a private test network of two full nodes. ### Getting Testnet Tokens In order to acquire sETH and sBZZ, refer to the [Fund Your Node](./../../bee/installation/fund-your-node.md) page. --- ## Store with Encryption In Swarm, all data is _public_ by default. To protect sensitive content, it must be encrypted so that only authorised users are able to decrypt and view the plaintext content. The Bee client can encrypt files and directories during upload, producing a reference that bundles a Swarm address with a decryption key. Only those in possession of the full reference can decrypt the content. ## Encrypt and Upload a File Include the `Swarm-Encrypt: true` header with your upload request: ```bash curl -F file=@bee.jpg \ -H "Swarm-Postage-Batch-Id: 78a26be9b42317fe6f0cbea3e47cbd0cf34f533db4e9c91cf92be40eb2968264" \ -H "Swarm-Encrypt: true" \ http://localhost:1633/bzz ``` More information on how to buy a postage stamp batch and get its batch id can be found [here](./buy-a-stamp-batch.md). When successful, Bee returns a 128-character (64-byte) reference instead of the usual 64-character (32-byte) unencrypted reference: ```json { "reference": "f7b1a45b70ee91d3dbfd98a2a692387f24db7279a9c96c447409e9205cf265baef29bf6aa294264762e33f6a18318562c86383dd8bfea2cec14fae08a8039bf3" } ``` The reference is composed of two 64-character (32-byte) parts: ``` f7b1a45b70ee91d3dbfd98a2a692387f24db7279a9c96c447409e9205cf265ba ← Swarm address (safe to share) ef29bf6aa294264762e33f6a18318562c86383dd8bfea2cec14fae08a8039bf3 ← decryption key (keep private) ``` The Swarm address (first 64 characters) is a standard content address — the same identifier you would get from an unencrypted upload and safe to share publicly. The decryption key (last 64 characters) is sensitive: anyone who holds the full 128-character reference can decrypt and read the original content. Access control is entirely possession-based — there is no server-side revocation. The key is never transmitted to the Swarm network; Bee only exposes it through the local API response. :::warning If you lose the decryption key portion of the reference, the encrypted data becomes permanently unrecoverable. Store the full 128-character reference in a secure location such as a password manager. ::: :::info Encryption is disabled by default on all Swarm gateways to protect your data. [Install Bee on your computer](./../../bee/installation/getting-started.md) to use the encryption feature. ::: ## Encrypt and Upload a Directory To upload an entire directory with encryption, package it as a tar archive and set the `Swarm-Collection: true` header: ```bash tar -cf site.tar ./my-website/ curl --data-binary @site.tar \ -H "Content-Type: application/x-tar" \ -H "Swarm-Collection: true" \ -H "Swarm-Postage-Batch-Id: 78a26be9b42317fe6f0cbea3e47cbd0cf34f533db4e9c91cf92be40eb2968264" \ -H "Swarm-Encrypt: true" \ http://localhost:1633/bzz ``` The response follows the same 128-character reference format. All files in the collection are encrypted with the same key, and the full reference is required to access any file within it. ## Download and Decrypt a File Supply the full 128-character reference to the `/bzz` endpoint. Bee downloads all the relevant chunks, decrypts them, and returns the original content: ```bash curl -OJ http://localhost:1633/bzz/f7b1a45b70ee91d3dbfd98a2a692387f24db7279a9c96c447409e9205cf265baef29bf6aa294264762e33f6a18318562c86383dd8bfea2cec14fae08a8039bf3 ``` :::danger Never use public gateways when requesting full encrypted references. The hash contains sensitive key information which should be kept private. Run [your own node](./../../bee/installation/getting-started.md) to use Bee's encryption features. ::: --- ## Ultra Light Nodes :::danger When running without a blockchain connection, bandwidth incentive payments (SWAP) cannot be made so there is a risk of getting blocklisted by other peers for unpaid services. ::: ## Configuration To run Bee as an ultra-light node `full-node` and `swap-enable` must both be set to `false`, and the `blockchain-rpc-endpoint` value should be set to an empty string `""` or commented out in the [configuration](./../bee/working-with-bee/configuration.md). ## Mode of Operation The target audience for this mode of operations are users who want to try out running a node but don't want to go through the hassle of blockchain onboarding. Ultra-light nodes will be able to download data as long as the data consumed does not exceed the payment threshold (`payment-threshold` in [configuration](./../bee/working-with-bee/configuration.md)) set by peers they connect to. Running Bee without a connected blockchain backend, however, imposes some limitations: - Can't do overlay verification - Can't do SWAP settlements Since we can't buy postage stamps: - Can't send PSS messages - Can't upload data to the network --- ## Upload & Download Uploading to Swarm has two steps: (1) **buy storage** as a **postage stamp batch** with a unique **batch ID**—and (2) **upload using the batch ID**. The upload returns a **Swarm reference hash**, anyone with that reference can download the content. :::info Example project The runnable Node.js scripts for this guide are in [`examples/upload-and-download`](https://github.com/ethersphere/examples/tree/main/upload-and-download). Clone the repo, copy `.env.example` to `.env`, fill in your `BEE_URL` and `BATCH_ID`, run `npm install`, then `npm run script:01` or `npm run script:02`. ::: **Before you begin:** - You need a running Bee node connected to Gnosis Chain and funded with **xBZZ** and **xDAI**. - Uploads always require a **postage stamp batch**. - Ultra-light nodes can download but **cannot upload**. ## Upload & Download with bee-js The `bee-js` library is the **official SDK for building Swarm-based applications**. It works in both **browser** and **Node.js** environments and **greatly simplifies development** compared with using the Bee HTTP API directly. It is the recommended method for developing applications on Swarm. Refer to the [`bee-js` documentation](https://bee-js.ethswarm.org/docs/) for more usage guides. :::tip **Environment-specific methods:** - **Browser-only:** [`uploadFiles`](https://bee-js.ethswarm.org/docs/api/classes/Bee/#uploadfiles) (multi-file via `File[]`/`FileList`) - **Node.js-only:** [`uploadFilesFromDirectory`](https://bee-js.ethswarm.org/docs/api/classes/Bee/#uploadfiles) (recursively reads local filesystem to upload multiple files in a directory using `fs`), - **Both:** [`uploadFile`](https://bee-js.ethswarm.org/docs/api/classes/Bee/#uploadfile) (with some environment specific usage), [`downloadFile`](https://bee-js.ethswarm.org/docs/api/classes/Bee/#downloadfile) ::: ### Single file — Node.js **Step-by-step Walkthrough:** 1. Create a Bee client: `const bee = new Bee("http://localhost:1633")` 2. Buy storage (postage stamp batch) by specifying storage size and duration: `const batchId = await bee.buyStorage(Size.fromGigabytes(1), Duration.fromDays(1))` 3. Read the file from disk: `const data = await readFile("./hello.txt")` 4. Upload bytes with filename & content type → get reference: `const { reference } = await bee.uploadFile(batchId, data, "hello.txt", { contentType: "text/plain" })` 5. Download the file by reference: `const file = await bee.downloadFile(reference)` 6. Log the downloaded file’s title and metadata: `console.log(file.name)` `console.log(file.contentType)` `console.log(file.data.toUtf8())` **Full example:** ```js // 1) Connect to your Bee node HTTP API const bee = new Bee("http://localhost:1633"); // 2) Buy storage (postage stamp batch) for this session const batchId = await bee.buyStorage( Size.fromGigabytes(1), Duration.fromDays(1) ); // 3) Read the file from disk as bytes const data = await readFile("./hello.txt"); // 4) Upload the bytes with a filename and content type; capture the reference const { reference } = await bee.uploadFile(batchId, data, "hello.txt", { contentType: "text/plain", }); console.log("Uploaded reference:", reference.toHex()); // 5) Download the file back using the reference const file = await bee.downloadFile(reference); // 6) Log the file's metadata and contents to the terminal console.log(file.name); // "hello.txt" console.log(file.contentType); // "text/plain" console.log(file.data.toUtf8()); // Prints file content ``` ### Single file — Browser :::info When working with browsers you can use the [`File` interface](https://developer.mozilla.org/en-US/docs/Web/API/File). The filename is taken from the `File` object itself, but can be overwritten through the second argument of the `uploadFile` function. ::: **Walkthrough** 1. Initialize a Bee object using the API endpoint of a Bee node: `const bee = new Bee("http://localhost:1633")` 2. Buy storage and get postage stamp batch ID: `const batchId = await bee.buyStorage(Size.fromGigabytes(1), Duration.fromDays(1))` 3. Create a `File` object: `const file = new File(["Hello Swarm!"], "hello.txt", { type: "text/plain" })` 4. Use batch ID to upload → get reference: `const { reference } = await bee.uploadFile(batchId, file)` 5. Download by reference: `const downloaded = await bee.downloadFile(reference)` 6. Log the downloaded file’s title and metadata: `console.log(downloaded.name) // "hello.txt"` `console.log(file.contentType) // "text/plain"` `console.log(downloaded.data.toUtf8()) // prints file content` ```js // 1) Connect to your Bee node HTTP API const bee = new Bee("http://localhost:1633"); // 2) Buy storage (postage stamp batch) for this session const batchId = await bee.buyStorage( Size.fromGigabytes(1), Duration.fromDays(1) ); console.log("Batch ID:", String(batchId)); // 3) Upload a single file created in code const file = new File(["Hello Swarm!"], "hello.txt", { type: "text/plain" }); const { reference } = await bee.uploadFile(batchId, file); console.log("Reference:", String(reference)); // 4) Download and print name + contents const downloaded = await bee.downloadFile(reference); console.log(downloaded.name); // "hello.txt" console.log(file.contentType); // "text/plain" console.log(downloaded.data.toUtf8()); // prints file content ``` ### Multiple files — Browser Use **`uploadFiles`** for multi-file upload in the browser. It accepts `File[]`/`FileList`. When using ``, each file’s **relative path** is preserved. To download a specific file later, pass the **collection reference** plus the **same relative path**. 1. Initialize a Bee object using the API endpoint of a Bee node: `const bee = new Bee("http://localhost:1633")` 2. Buy storage and get postage stamp batch ID: `const batchId = await bee.buyStorage(Size.fromGigabytes(1), Duration.fromDays(1))` 3. Create files for upload: ```js const files = [ new File(["Hello Swarm"], "index.html", { type: "text/html" }), new File(["body{font-family:sans-serif}"], "assets/main.css", { type: "text/css", }), ]; ``` 4. Upload multiple files (collection) → get collection reference: `const res = await bee.uploadFiles(batchId, files)` 5. Download files by relative paths: `const logo = await bee.downloadFile(res.reference, "images/logo.png")` 6. Log the downloaded file’s title and contents: `console.log(page.name) // "index.html"` `console.log(page.data.toUtf8()) // prints file content` ```js // 1. Initialize a Bee object const bee = new Bee("http://localhost:1633"); // 2. Buy storage and get batch ID const batchId = await bee.buyStorage( Size.fromGigabytes(1), Duration.fromDays(1) ); console.log("Batch ID:", String(batchId)); // 3. Create files for upload const files = [ new File(["Hello Swarm"], "index.html", { type: "text/html" }), new File(["body{font-family:sans-serif}"], "assets/main.css", { type: "text/css", }), ]; // 4. Upload multiple files (collection) → get collection reference const res = await bee.uploadFiles(batchId, files); console.log("Collection ref:", String(res.reference)); // 5. Download files by relative path const page = await bee.downloadFile(res.reference, "index.html"); console.log(page.name); // "index.html" console.log(page.data.toUtf8()); // prints file content const style = await bee.downloadFile(res.reference, "assets/main.css"); console.log(style.name); // "main.css" console.log(style.data.toUtf8()); // prints file content ``` ### Multiple files — Node.js **Step-by-step Walkthrough:** 1. Initialize a Bee object using the API endpoint of a Bee node: `const bee = new Bee("http://localhost:1633")` 2. Buy storage and get postage stamp batch ID: `const batchId = await bee.buyStorage(Size.fromGigabytes(1), Duration.fromDays(1))` 3. Recursively upload a local directory → get collection reference: `const res = await bee.uploadFilesFromDirectory(batchId, "./site")` 4. Download one file by its relative path: `const page = await bee.downloadFile(res.reference, "index.html")` 5. Log the downloaded file name and contents: `console.log(page.name ?? "index.html")` `console.log(page.data.toUtf8())` **Full example:** ```js // 1) Connect to your Bee node HTTP API const bee = new Bee("http://localhost:1633"); // 2) Buy storage (postage stamp batch) const batchId = await bee.buyStorage( Size.fromGigabytes(1), Duration.fromDays(1) ); // 3) Upload all files under ./files (relative paths preserved); get reference const res = await bee.uploadFilesFromDirectory(batchId, "./files"); console.log("Directory uploaded. Collection reference:", res.reference.toHex()); // 4) Download files from the collection by original relative paths const page = await bee.downloadFile(res.reference, "root.txt"); const stylesheet = await bee.downloadFile( res.reference, "subdirectory/example.txt" ); // 5) Log the file name and contents to the terminal console.log(page.name); // "root.txt" console.log(page.data.toUtf8()); // prints file content console.log(stylesheet.name); // "example.txt" console.log(stylesheet.data.toUtf8()); // prints file content ``` ## Upload & Download with the Bee API (advanced) The **Bee HTTP API** offers the **lowest-level access** to a Bee node. It is **more complex and harder to use** than **bee-js** because you must manage headers, content types, and postage parameters yourself. **Unless you specifically require raw HTTP control**, we **do not recommend** using the Bee API directly — use **bee-js** instead for application development. Refer to the [Bee API reference specification](https://docs.ethswarm.org/api/) for detailed usage information. The Bee API exposes three HTTP endpoints: - **`/bzz`** — upload & download files/directories (most common) - **`/bytes`** — upload & download raw data - **`/chunks`** — upload & download individual chunks ### Upload with **/bzz** While `bee-js` allows postage stamp batches to be purchased by specifying storage duration and data size, the raw Bee API requires `amount` and `depth` parameters directly. The relationship between these parameters and the storage size and duration of the batch is complex, so `bee-js` is strongly encouraged for newcomers. [Learn more](./tools-and-features/buy-a-stamp-batch.md). 1. Buy a postage batch: ```bash curl -s -X POST http://localhost:1633/stamps// ``` 2. Upload a file with the returned `batchID`: ```bash curl -X POST \ -H "Swarm-Postage-Batch-Id: " \ -H "Content-Type: text/plain" \ --data-binary "@test.txt" \ http://localhost:1633/bzz ``` Response: ```json { "reference": "22cbb9cedca08ca8d50b0319a32016174ceb8fbaa452ca5f0a77b804109baa00" } ``` 3. Download with `/bzz` ```bash curl http://localhost:1633/bzz/ -o output.txt ``` --- **Next:** [Host a Webpage](/docs/develop/host-your-website) — upload a static website and serve it through `/bzz//`. --- ## Access Content Accessing content on Swarm using Swarm Desktop is easy. All you need to get started is the Swarm hash for the content you wish to access. Whenever content is [uploaded to Swarm](./upload-content.md) a Swarm hash is generated as a reference to that content. To access content on Swarm go to the ***Files*** tab and click ***Download***: ![](/img/access1.png) From there, paste the Swarm hash for the content you want to access, and click ***Find***. We'll use the hash for a Swarm blog post explaining how to upload a website to Swarm: `bc9b942212421e2a19fe1ffdf0add641ae530923041ea8f549381747b14b2f2d` ![](/img/access2.png) On the following screen you will see the data associated with the Swarm hash and see options for downloading (or browsing if it is a hash for a website): ![](/img/access3.png) Click ***View Website*** to see the site in your browser, or ***Download*** to download the files: ![](/img/access4.png) --- ## Backup and Restore ## Create a Backup To create a backup of your Bee node in Swarm Desktop, start by shutting down your node. Right click the Bee icon in the System tray and select `Stop Bee` and then `Quit` to close and exit from Swarm Desktop: ![](/img/backup2.png) Next navigate to the `Settings` tab in the app and copy the location of the data directory as indicated in the `Data DIR` field: ![](/img/backup1.png) Navigate to the directory you just copied and create copies of all the files in that directory (`\data-dir`), including `localstore`, `statestore`, `stamperstore`, `kademlia-metrics` and `keys` folders and store them in a secure and private location. ![](/img/backup7.png) In addition to the data folders, you will also need the password found in the `config.yaml` file in order to restore a Bee node from backup. Move up one directory from `Data DIR` to the `Data` directory, and create a copy of the `config.yaml` file and save it along with the other folders you just backed up: ![](/img/backup4.png) Alternatively you may open the `config.yaml` and save the password as a text file along with the rest of your backup files: ![](/img/backup5.png) Your completed backup should contain all the files from your data directory as well as your password (either in your `config.yaml` file or as a separate file or written down.) ![](/img/backup8.png) ### Back-up Gnosis Chain Key Only If you only wish to back-up your Gnosis Chain key, navigate to the `\data-dir\keys` directory, and copy the `swarm.key` to a safe location: ![](/img/backup9.png) You also need the password found in the `config.yaml` file in order to access your Gnosis Chain account. Move up one directory from `Data DIR` to the `Data` directory, and create a copy of the `config.yaml` file and save it along with the other folders you just backed up: ![](/img/backup4.png) Alternatively you may open the `config.yaml` and save the password as a text file along with the rest of your backup files: ![](/img/backup5.png) ## Restore from Backup To restore from backup, begin with a [new install](./install.md) of Swarm Desktop. Once the installation process is finished, navigate to the `Settings` tab in the app and copy the install file directory as indicated in the `Data DIR` field: ![](/img/backup1.png) Before navigating to the directory you just copied, right click the Bee icon in the System tray and select `Stop Bee` and then `Quit` to close and exit from Swarm Desktop: ![](/img/backup2.png) Next open your file explorer and navigate to the directory you just copied. Delete any files present in the directory, and replace them with your own backup copies (excluding the `config.yaml` / password file): ![](/img/backup7.png) Move up one directory from `Data DIR` to `Data`, and replace delete the `config.yaml` file and replace it with the `config.yaml` file from your backup. Alternatively if you have saved just the password and not the entire config file, open the default `config.yaml` file in a text editor such as VS Code or a plain text editor: ![](/img/backup4.png) ![](/img/backup5.png) Replace the `password` string with your own password which you saved from the `config.yaml` backup. Restart Swarm Desktop and check to see if the backup was restored successfully: ![](/img/backup6.png) ### Restore Gnosis Chain Account If you only wish to access your Gnosis Chain account, you can [follow these instructions](./../bee/working-with-bee/backups.md#metamask-import) for exporting to Metamask in order to access your account. --- ## Configuration(Desktop) ## Setting RPC Endpoint In order to interact with the Gnosis Chain to buy stamps, participate in staking, and manage assets such as xBZZ, Bee nodes require a valid Gnosis Chain RPC endpoint. By default the RPC endpoint is set to https://xdai.fairdatasociety.org, however any valid Gnosis Chain RPC endpoint may be used. To modify the RPC endpoint, first navigate to the ***Settings*** tab: ![](/img/config1.png) From the ***Settings*** tab, expand the API Settings section and click the pen button next to Blockchain RPC URL to edit the default RPC. You can choose any valid Gnosis Chain RPC, either from your own Gnosis node or a service provider. You can find a list of paid and free RPC options from the [Gnosis Chain docs](https://docs.gnosischain.com/tools/RPC%20Providers/). For this example we will use the free endpoint - *https://xdai.fairdatasociety.org*. :::warning Other ***free public RPC endpoints are discouraged,*** since they may enforce rate limiting or may not store the historical smart contract data required by Bee nodes. [Read more](./../bee/working-with-bee/configuration.md#setting-blockchain-rpc-endpoint). ::: ![](/img/config2.png) Click ***Save and Restart*** to finish changing the RPC endpoint. ## Upgrading from an Ultra-light to a Light Node Bee ultra-light nodes are limited to only downloading small amounts of data from Swarm. In order to download greater amounts of data or to upload data to Swarm you must upgrade to a light node. To do this we need to first fund our Swarm Desktop Bee node with some xDAI (DAI bridged from Ethereum to Gnosis Chain which serves as Gnosis Chain's native token for paying transaction fees) in order to pay for the Gnosis Chain transactions required for setting up a light node. ### Bridging Ethereum DAI to Gnosis Chain as xDAI If you already have some xDAI on a Gnosis Chain address, skip to the next step ***Funding Node with xDAI***. If you have DAI on Ethereum and need to swap it for xDAI, you can use one of the [Gnosis Chain Bridge](https://bridge.gnosischain.com/) Five to ten xDAI is plenty to get started. ### Funding Node with xDAI Once you have a few xDAI in your Gnosis Chain address, to fund your Bee node you need to send it from your wallet to your Swarm Desktop wallet. You can find your address from the ***Account*** tab of the app. ![](/img/config3.png) Next simply send your xDAI to that address. Before sending, make sure you have set your wallet to use the Gnosis Chain network and not the Ethereum mainnet. If Gnosis Chain is not included as default selectable network in your wallet, you may need to add the network manually. You can use this configuration to add Gnosis Chain: | Field | Value | |--------------|-----------| |**Network name:**|Gnosis| | **New RPC URL:** | https://xdai.fairdatasociety.org | | **Chain ID:**| 100 | | **Symbol:**| xDai | | **Block Explorer URL (Optional):**| https://gnosis.blockscout.com/ | ![](/img/config4.png) The transaction should be confirmed in under a minute. We can check on the ***Account*** page to see when the xDAI has been received: ![](/img/config5.png) ### Set Up Wallet Now with some xDAI in the Swarm Desktop wallet, we can upgrade our Bee node from ultra-light to a light node. Completing the setup process will swap xDAI for some xBZZ at the current price, and will issue the transactions needed to set up the chequebook contract. To get started, navigate to the ***Info*** tab and click the ***Setup wallet*** button. ![](/img/config10.png) Click ***Use xDAI***. ![](/img/config6.png) Confirm that you have sufficient xDAI balance and click ***Proceed***. ![](/img/config7.png) Click ***Swap Now and Upgrade***. ![](/img/config8.png) Wait for the upgrade to complete. ![](/img/config9.png) After the upgrade is complete, you will see several new sections within the ***Account*** tab: ***Chequebook***, ***Stamps***, and ***Feeds***. ## Fund Chequebook After setting up your wallet you will have access to the ***Chequebook*** section from the ***Accounts*** tab. From here you can manage your chequebook for your Swarm Desktop Bee node. --- ## Install ## Download and Install Swarm Desktop Installing the Swarm Desktop app takes only a few clicks. To get started, simply download and install the Swarm Desktop app for your operating system. Installers are available for Windows, Linux, and OSX. You can find download links for Swarm Desktop at the Swarm [homepage](https://www.ethswarm.org/build/desktop) and you can find installers for specific operating systems at the [releases page](https://github.com/ethersphere/swarm-desktop/releases) of the Swarm Desktop GitHub repo. :::caution Swarm Desktop is in Beta and currently includes the Sentry application monitoring and bug reporting software which automatically collects data in order to help improve the software. ::: :::caution This project is in beta state. There might (and most probably will) be changes in the future to its API and working. Also, no guarantees can be made about its stability, efficiency, and security at this stage. ::: [![](/img/desktop-homepage-dl.png)](https://www.ethswarm.org/build/desktop) *Ethswarm.org Swarm Desktop Page* [![](/img/desktop-releases-dl.png)](https://github.com/ethersphere/swarm-desktop/releases) *Swarm Desktop GitHub Releases Page* After running the installer, a window will pop up and display the installation status: ![](/img/desktop-install-downloading.png) Once the installation is complete, Swarm Desktop will open up in your default browser in a new window to the "Info" tab of the app: ![](/img/desktop-new-install.png) If the installation went smoothly, you should see the message "Your node is connected" above the "Access Content" button along with a status message of "Node OK". ### What Just Happened? Running the Swarm Desktop app for the first time set up a new Bee node on your system. The installation process generated and saved private keys for your node in the Swarm Desktop's data directory. Those keys were used to start up a new Bee node in ultra-light mode. :::warning If your Swarm Desktop files are accidentally deleted or become corrupted you will lose access to any assets or data which are secured using those keys. Make sure to [backup your keys](./backup-restore.md). ::: ### "Ultra-light" and "Light" Nodes Swarm Desktop by default starts up a node in "ultra-light" mode. When running in ultra-light mode Swarm Desktop limited to only downloading data from Swarm. Moreover, it's limited to downloading only within the free threshold allowed by other nodes. For instructions on switching to light mode see the [configuration section](./configuration.md). ## Tour of Swarm Desktop ### Info Tab The "Info" tab gives you a quick view of your Swarm Desktop's status. From here we can quickly see if the node is connected to Swarm, whether the node is funded, and whether its chequebook contract is set up. On a new install of Swarm Desktop, the node should be connected, but the wallet and chequebook will not have been set up yet. ![](/img/swarm-desktop-info-tab.png) ### Files Tab From "Files" tab you can input a Swarm hash in order to download the file associated with the hash. See this full [guide for downloading](./access-content.md) using Swarm Desktop. ![](/img/swarm-desktop-files-tab.png) ### Account Tab From the "Account" tab you can view your Swarm Desktop node's Gnosis Chain address and associated xBZZ and xDAI balances. ![](/img/swarm-desktop-account-tab.png) ### Settings Tab From the "Settings" tab you can view important settings values. Note that the Blockchain RPC URL and ENS resolver URL are already filled in, and only the Blockchain RPC URL is modifiable through this tab. If you wish to modify other settings see the [ configuration page](./configuration.md) for detailed instructions. ![](/img/swarm-desktop-settings-tab.png) ### Status Tab From the "Status" tab you can see a quick overview of the health of your Swarm Desktop's Bee node. ![](/img/swarm-desktop-status-tab.png) --- ## Introduction(Desktop) ![](/img/swarm-desktop.png) The Swarm Desktop app provides an easy-to-use graphical user interface for running a Bee node and interacting seamlessly with the Swarm network. While running Bee from the terminal is a powerful and flexible approach for developers and node operators, the Swarm Desktop app is a simpler alternative for more basic use cases. The Swarm Desktop App was designed to simplify the Swarm onboarding process so that anyone can benefit from decentralized storage while maintaining privacy and control over their data. Available for Windows, Mac, and Linux operating systems, the Swarm Desktop App serves as a personal gateway to the Swarm network. --- ## Postage Stamps(Desktop) :::info Swarm Desktop must be configured as a light node in order to access stamp related features. If you have not already upgraded from the default ultra-light configuration, complete the upgrade by following the ***[instructions here](./configuration.md#upgrading-from-an-ultra-light-to-a-light-node)***. ::: Postage stamps are required in order to upload data to Swarm. Postage stamps are purchased by interacting with the Swarm postage stamp smart contract on Gnosis Chain. Postage stamps are not purchased one by one, rather they are purchased in batches only. ## How to Buy a Postage Stamp Batch Stamps can be purchased by selecting ***Stamps*** from the ***Account*** tab: ![](/img/stamps1.png) And then clicking the ***Buy New Postage Stamp*** button: ![](/img/stamps2.png) ### Depth and Amount Batch [depth and amount](./../concepts/incentives/postage-stamps.md) are the two required parameters which must be set when purchasing a postage stamp batch. Depth determines how many chunks can be stamped with a batch while amount determines how much xBZZ is assigned per chunk. ![](/img/stamps3.png) Inputting a value for depth allows you to preview the upper limit of data which can be uploaded for that depth. Inputting a value for amount and depth together will allow you to also preview the total cost of the postage stamp batch as well as the TTL (time to live - how long the batch can store data on Swarm). Click the ***Buy New Stamp*** button to purchase the stamp batch. ![](/img/stamps4.png) After purchasing stamps you can view stamp details from the ***Postage Stamps*** drop down menu: ![](/img/stamps5.png) ## Managing Postage Batches After purchasing a postage batch, it is important to monitor the usage and TTL (time to live) of your batch. TTL is shown next to the "Expired in" label in the screenshot below. ![](/img/stamps6.png) For this stamp batch, it has only 6 hours left. Once the TTL has run out completely, the content uploaded using that batch will no longer be kept on Swarm, and will be lost forever. To prevent this from happening, you can "top up" your batch by adding more xBZZ to the batch balance to increase the batch TTL. ## Top-up a Batch To get started, click on the "Topup and Dilute" button. ![](/img/stamps7.png) From the "Action" dropdown menu, make sure that you have "Topup" selected and then fill in the `amount` by which you wish to top up the batch. Note that the number entered here is in PLUR (1e-16 xBZZ), and it is the same `amount`` parameter described in the [section above](./postage-stamps.md#depth-and-amount) on purchasing postage stamp batches, it is NOT equal to the total amount of xBZZ spent for this top up transaction. After inputting the `amount`, click "Topup" to submit the transaction. After a few moments, you will see a notice that the transaction was successful in a green alert box. A few moments after that, you will see the updated TTL in the stamp details window. ![](/img/stamps8.png) ## Dilute a Batch If our batch begins to come close to becoming fully utilised, we can choose to increase the `depth` of the batch to increase the amount of data it can store. This is referred to as "dilution", since by increasing the `depth` without updating the `amount`, we dilute the amount of xBZZ which is assigned to each chunk. In other words, the dilute transaction will increase the amount which can be uploaded by a batch while also ***decreasing*** the TTL. Therefore it is important to both top up and also dilute your stamp batch if you wish to increase the amount stored by the batch without decreasing its TTL. To get started, click on the "Topup and Dilute" button. Make sure to select "Dilute" from the "Action" dropdown menu. ![](/img/stamps9.png) From here, we can select the new `depth` value for our postage stamp batch. In this instance, we will increase it from 20 to 21. ![](/img/stamps10.png) After a few moments the transaction will be completed and you should see the updated Depth, Capacity, and TTL. ![](/img/stamps10.png) Note that both the Depth and Capacity have increased while the TTL has decreased. --- ## Publish a Website ## Step by Step Guide ### Install Swarm Desktop and Deposit Funds First, download and [install the Swarm Desktop App](./install.md). Next, add xDAI (transaction fees) to your Node Wallet address. If you possess xBZZ (storage fees), you can deposit it alongside the xDAI. Otherwise, you can exchange your xDAI for xBZZ using the Swarm Desktop app. Follow these steps to deposit funds: 1. Launch the Swarm Desktop App and go to the Account section in the left menu. 2. Transfer xDAI to your node wallet address. For safety, we suggest sending no more than 5 to 10 xDAI. 3. After funding your wallet, click the Top Up Wallet button on the right side of the screen. 4. Select the Use xDAI option. 5. Verify your xDAI balance and click Proceed. 6. Specify the amount of xDAI to convert to xBZZ and click Swap Now. 7. Your Node Wallet address will be credited with xBZZ. ![](/img/upload-a-website1.gif) ### Setup Chequebook Your node address is now funded with xDAI and xBZZ. However, to upload data on Swarm, you will need to transfer your funds to the Chequebook contract address. Follow these steps: 1. Go to the Account section in the left menu. 2. Select the Chequebook tab in the top menu. 3. Click the Deposit button. 4. Specify the amount of xBZZ to deposit into your Chequebook, which will be used for storage costs. ### Publish Website To publish your website on Swarm, follow these steps: 1. Navigate to the Files tab. 2. Click the Add Website button. 3. Select your website folder. NOTE: The index.html file should be in the root folder. 4. Purchase a Postage Stamp to publish your page. NOTE: Postage stamps cover storage costs for a specified duration. 5. Upload the website. ![](/img/upload-a-website2.gif) Once uploaded, your website can be accessed through its Swarm hash via your local Bee node or through a public gateway. Sharing the hash is a convenient way to distribute your content to users who aren't running their own Bee node—they can access it directly through any Swarm gateway. Your website is now accessible via: **Local Bee node:** ``` http://localhost:1633/bzz/bc9b942212421e2a19fe1ffdf0add641ae530923041ea8f549381747b14b2f2d/ ``` **Public gateway:** ``` https://api.gateway.ethswarm.org/bzz/bc9b942212421e2a19fe1ffdf0add641ae530923041ea8f549381747b14b2f2d/ ``` Replace the hash with your actual website hash. ### Connecting an ENS Domain to Your Website Associating your ENS domain with a Swarm hash generates a memorable, user-friendly identifier for your website, allowing users to easily locate and access your website without having to recall a lengthy, complex Swarm hash. Initially, you’ll need to register your domain name. To register and manage your ENS domain, you can use the ENS Domains Dapp along with the MetaMask browser extension. After registering your name and connecting MetaMask to the relevant Ethereum account, set the resolver to use the public ENS if you haven’t already. 1. Navigate to My Names and select the name you want to link to your Swarm content. 2. Click on ADD/EDIT RECORD. 3. From the "add record" dropdown menu, select Content. 4. Enter your Swarm Hash, beginning with "bzz://" and click "Save." ![](/img/upload-a-website3.gif) Your website is now available on: [https://api.gateway.ethswarm.org/bzz/swarm-devrel.eth/](https://api.gateway.ethswarm.org/bzz/swarm-devrel.eth/) ### Update the Website: Set up and update a feed Swarm feeds allow you to easily create a permanent address for your content stored on Swarm that you can update at any time. If you plan to update your website in the future, it’s recommended that you set up a “Feed” before uploading your website to Swarm. This way, the Swarm Hash connected to your ENS domain will stay the same, even as you change the content behind that hash. This will enable you to update your website’s content without changing the Swarm Hash and incurring Ethereum transaction costs each time you do so. #### Set up a Feed: 1. Navigate to to Account 2. Click on Feeds in the top menu 3. Click on Create New Feed 4. Define Identity name 5. And click Create Feed. #### Upload Website on Swarm and connect it to the Feed: 1. Navigate to to Account 2. Click on Feeds in the top menu 3. Choose the Feed you want to update 4. Click View Feed Page 5. Click the Add Website button. 6. Select your website folder. NOTE: The index.html file should be in the root folder. 7. Add Postage Stamp to publish your page. NOTE: Postage stamps cover storage costs for a specified duration. 8. Upload the website to your Node. 9. Connect the Feed hash to your ENS domain using the ENS steps shown earlier. ![](/img/upload-a-website4.gif) By following these instructions, you can now leverage the benefits of decentralised storage, maintain a censorship-resistant website, and create a user-friendly experience by connecting your site to an ENS domain. --- ## Start a Blog ## A Guide to Starting Your Blog on Swarm There are many different approaches to starting a blog on Swarm, however the easiest is to use the Etherjot Web blogging tool. Etherjot Web is a straightforward tool for publishing and editing your blog on Swarm. It handles all uploading of files, page customization, basic UI template, and even comes with a "comments" feature so any other Swarm user can leave a comment on your blog. ## Requirements * [Swarm Desktop](./install.md) with a [valid postage stamp batch](./postage-stamps.md) ## Getting Started To get started you must first have installed Swarm Desktop and have it running on your computer with a [valid stamp batch](./postage-stamps.md). Note that your blog will only stay online as long as the postage batch is still valid, therefore you must make sure to stay aware of the postage batch TTL (time to live), and [top up your batch](./postage-stamps.md#top-up-a-batch) regularly in order to keep your content online. ### Open Etherjot To open Etherjot, right click the Swarm Desktop icon in your dashboard and navigate to "Apps", and then click on "Etherjot". ![](/img/etherjot27.png) ## Initialize Your Blog When first starting Etherjot Web, you will be greeted with this page: ![](/img/etherjot1.png) On this page, as long as you have fulfilled the requirements outlined above, you will see two green checkmarks confirming you have Swarm Desktop running with a valid postage stamp batch. You will also see a warning reminding you of the importance of [topping up your stamp batch](./postage-stamps.md#top-up-a-batch) to prevent the batch TTL from running out. :::danger In addition to monitoring your postage stamp batch TTL, it is also important that you back up your blog, or else you may lose access to your blog in Etherjot (although it will still remain live on Swarm as long as its stamp batch has not expired). ::: Fill in your blog name, check the box with the TTL warning, and click the "Create" button to initialize your blog. This will issue a Swarm transaction to set up a feed for your blog. The transaction will take a few moments, after which you will be greeted with the Etherjot Web blog editor. ![](/img/etherjot17.png) The "Swarm Hash" displayed at the top of the editor is the address for the homepage of your blog. Click "Open" to navigate to your blog. We can see now that the blog has been initialized, but no content has been uploaded. ![](/img/etherjot18.png) ## Don't Lose Your Work! Due to the decentralised nature of Swarm and applications built on Swarm, there are several precautions you should take which you may be unfamiliar with when coming from a Web 2.0 application. ### Back-up Your Blog No username and password are required for editing your blog and uploading new posts. However, you do need to make sure to back up your blog in order to prevent losing access to it. You should do this after initializing your blog, and you should also back up your blog again after publishing any changes. To back up your blog, start by clicking "Settings." ![](/img/etherjot21.png) From the Settings page, click "Export." ![](/img/etherjot22.png) Copy the displayed text to a `.json` file, make certain to copy the entire displayed text. This is your backup file and is used to import your blog. Note that the backup contains the private key of your blog, so should not be revealed to anyone else. ![](/img/etherjot23.png) ### Avoid Losing Changes (DANGER) Etherjot currently does not allow you to save drafts locally, so if you navigate away from the blog post you are currently editing, you will lose any changes you have made which have not yet been uploaded to Swarm. Take note of the three UI elements highlighted in the screenshot - using the "+" or "Settings" buttons will cause you to lose any changes not uploaded to Swarm, and hitting the "Reset" button will cause you to lose everything which has not been backed up. :::danger Hitting the "Reset" button will cause you to lose any content which has not yet been published and [backed up](./start-a-blog.md#back-up-your-blog). ::: ![](/img/etherjot19.png) If you click the "+" button or the "Settings" button, you will see a warning to notify you that any unsaved changes will be lost. ![](/img/etherjot20.png) You will NOT see a warning for refreshing your browser page, however, so be careful not to refresh your browser before publishing any changes to Swarm. ## Writing Your Blog ### Add Some Text The text editor for your blog has two main panels. The one on the left is where you can write your content using [Markdown](https://www.markdownguide.org/). On the right side is where you can see a preview of your rendered markdown as it will appear to a visitor to your blog. ![](/img/etherjot3.png) Let's fill in some content and examine the preview. ![](/img/etherjot4.png) Here you can see the new content we just wrote, note that there is no auto-save functionality, so any changes we make will not be saved until we click "Publish" to upload the changes to Swarm. However you will see that the "Publish" button is greyed currently, as we have not yet filled in all the required fields for publishing. ### Add Media Files Next let's try to add an image. To get started, we need to click the "Asset Browser" button. ![](/img/etherjot2.png) This will open up the Asset Browser where you can manage your blog assets such as images. ![](/img/etherjot5.png) To upload your file, click the "browse" button and choose the file you wish to upload. :::info In addition to images, video and audio files may also be uploaded, however currently the URL to the Swarm hash must be manually inserted into html `