Maidsafe coin Details+Update



  • MaidSafe Dev Update - Test 18

    Today, we are pleased to be releasing Test 18 for all forum users who are at least trust level 1

    This is a new network, so trying to access data from Test 17 won’t work (you need to create a new account). We are planning to take Test 17 offline this weekend (both the internal test and Test 17 itself).

    Here are the main changes compared to Test 17

    • The storage limit has been doubled from 500 operations (store and amend) to 1000 per account.
    • Add new FFI function to get the account info from the network (number of available/used mutations)
    • Implement operations recovery for account creation, app authentication and app revocation.
    • Modify the client Rate Limiter paradigm to not put an upper threshold on number of clients with unique IPs to bootstrap off a proxy.

    It’s also now much easier to run your own local network (because of the
    new developer configuration options that have been added to Routing,
    Crust and Vault). See this Dev Forum topic for more info.

    Earlier this week, we started to document how to use the mock, local and
    actual network (for developers who want to build SAFE Network
    applications and websites). See this Dev Forum topic to read our first draft.

    For Test 18, we are intending to have some front-end updates next week
    to introduce an option for apps to request access to arbitrary Mutable
    Data objects (see this JIRA task for more details). The network itself shouldn’t require an update (unless something unforeseen occurs of course).

    Based on the feedback from the community, we are redesigning the UI for Web Hosting Manager app. You can see the new design in this Dev Forum topic. Please do share your feedback on the latest design.

    SAFE Browser v0.4.0

    To connect to Test 18, you need to use SAFE Browser v0.4.0.

    Accounts can be created using the authenticator bundled with the SAFE Browser. To open it, click on the Authenticator icon in the toolbar of the browser.

    This testnet is limited to forum users who are at least trust level 1 See this topic11 for more information.

    To obtain an invitation token, trust level 1 forum users can use this website20, which can also be accessed via the SAFE Browser itself when creating an account.

    Each client account is limited to 1000 operations. It’s worth noting that this number is just a temporary placeholder and is shared between all the apps on a given account.

    Please be aware that we might need to restart this test network, wiping all data that is stored on it. We will announce this in a dev update (when possible) prior to taking it down.

    Download

    Changelog

    • Freeing of safe_app object handles. Object handles created by the applications will be automatically freed on closing the tab or even on refresh.
    • Toggle safe browsing. Allow browsing of clearnet sites by toggling from File > SAFE Browsing Enabled
    • Index file to be read automatically. Providing the <service>.<publicid> as the URL loads the index.html file of the service.
    • The number of store/modify operations completed for the account is displayed in the Authenticator.
    • Enhancements to the NFS API are now being exposed to the DOM layer for web applications.
    • safe: protocol URLs will be opened in the SAFE Browser when opened from external links.
    • localhost: protocol is now supported. In order to open a URL like localhost:<port>, the URL must be converted to localhost://<anything>:<port>, where <anything> can be any characters but not empty, for example, localhost://p:3003/.
    • Several UI enhancements and minor fixes have been introduced, like ordering the application list, reducing the header size, loading overlay has rounded edges.
    • Issues with the building process for mock and real network have been resolved.
    • Compatible with Test 18.

    Known limitation

    • History and bookmarks are not saved in the network right now.

    Example applications

    Known limitation

    • Public IDs created via examples are not exchangeable between the apps.
      • This is the error reported with the Access Denied / Requested Entry not found error. This issue is related to the permissions of the service/mail mutable data. The root folder with the default folders are designed for easier sharing and collaboration between apps. Applications can request access to these while the authenticator manages the app access levels. In the case of public IDs, the public ID is added as a key into the _publicNames container. A corresponding MutableData is created by the app and the permissions for the same are set by the app. This MutableData is then added as the value to the public ID entry. Now when the second app tries to add a service to the MutableData created by the first app, it doesn’t have the permissions to do so as this was not created by the authenticator. To resolve this, we are going to introduce an option for apps to request access to arbitrary Mutable Data objects (see this JIRA task9 for more details).

    Web Hosting Manager v0.1.2

    When uploading files using Web Hosting Manager, the maximum file size is 25 MB per file.

    Download

    Changelog

    • Present more user-friendly error messages.
    • Long public ID and service name are truncated and shown on the UI.
    • Internal use of the NFS API has been adapted to recent changes in safe_client_libs master branch. The NFS API was using ImmutableData to store the files and now the API is updated to save DataMap pointers. For each file, a pointer to the data map is stored in the MutableData entry representing that file. The actual data map is stored in the network either encrypted or plaintext depending on the privacy level specified.
    • Window resizing is now disabled.
    • Compatible with Test 18.

    SAFE Mail Tutorial v0.2.2


    Download

    Changelog

    • Compatible with Test 18.

    SAFE Web API Playground

    A new tool called SAFE Web API Playground is now being formally maintained in MaidSafe’s safe_examples repository6, which allows any developer to learn and experiment with the DOM API in a friendly and interactive way.

    It is meant to help people aiming at creating websites/webapps that need to access the SAFE Network DOM API, which is available on the global window object in the SAFE Browser.

    Instead of having to upload your site to the network every time you want to test some code, you can simply use this tool to experiment and get a deeper understanding of how each API works.

    In order to use it, follow the instructions found in its README, and either upload it as a web app using the Web Hosting Manager application or just run it locally and access it from the SAFE Browser using the now supported localhost protocol (see above for details).

    It is also available on the test network at safe://webapi.learn/, however, it is much faster when accessed locally.

    Repository

    Clone it from GitHub



    Example HTML files

    An examples.html file has been published with the safe_examples release. This file showcases some basic use cases of the DOM API, like authorising an application, creating MutableData objects, as well as reading and iterating over MutableData entries. You can upload this file using the web_hosting_manager application and publish it as a website, and then run any of the three examples available from the SAFE Browser.

    Another file called leaking.html has been also published with the safe_examples release. This file can be used for testing the browser capability of freeing safe_app object handles. If you click on one of the example applications it will create a client and authorise the app but it won’t free them, causing a leak in clients. After running any of these examples more than twice you will start getting a connection error due to the limit in the number of clients being reached. Refreshing the page will automatically free all leaked objects allowing you to run any of the examples again.

    Support

    If you need help with anything related to SAFE Browser, Web Hosting Manager or SAFE Mail Tutorial, please use the #support category of this forum.

    Where should I report issues?

    If you know which component a bug is associated with, feel free to report it on GitHub in the corresponding repo.


    SAFE Authenticator & API

    • Upgrade safe_client_libs to latest in master branch.
    • Support the new NFS API from safe_client_libs. The safe_app_nodejs library was internally using ImmutableData to store the files, but this is now being handled directly by safe_client_libs.
    • Allow the logPath function in the API to receive a filename parameter to automatically generate the log path for it. The path will be the same as the path of the safe_core lib log file.
    • The appToken parameter in the DOM API functions was renamed to appHandle to make it more consistent with what it really is.

    SAFE Client Libs

    The pull request for operations recovery in safe_authenticator has been merged now, but this road doesn’t end here and we are planning on several upcoming improvements and fixes for this feature - all of them required to be sure that we have covered most of the known edge cases and this feature works as intended. One of the possible limitations is the concurrent app revocation: for instance, if a user wants to revoke apps on their PC and a mobile device at the same time, it’s guaranteed by the design that nothing wrong should happen, but we want to verify it and we’ll be improving our test suite. Also, we already have a plan to improve, optimise, and simplify some things related to how apps authentication and revocation work, so that’s what we’ll be working on for the rest of this week. However, as these changes are mostly just nice-to-haves and the main functionality is implemented & tested, we’re releasing new versions of the SAFE Client Libs crates with what we already have.

    The team also made a few additional changes. @marcin@canndrew has finished his work on the new feature to share Mutable Data objects between apps through the Authenticator, and his pull request will be merged soon. @adam has updated the repository to use the recent stable Rust version (1.19).

    We’ve also been involved in debugging an interesting bug that was
    reported by the front-end team as we weren’t sure of its origin (on the
    Rust side or on the front-end). As it turned out, it was neither: we’ve
    traced it down to node-ffi (the library that we use to communicate
    between JavaScript/Node.js and Rust) and discovered that this library blocks the execution
    thread while waiting for a callback to return a result. This comes into
    a conflict with our rule about non-blocking front-end callback
    functions and this behaviour caused SAFE Browser freezing in certain
    circumstances. We’ve tracked down & fixed these particular cases but
    the problem still remains and soon we’ll be discussing how to fix it
    properly.

    Routing & Vault

    The Local Network support via dev options is now fully integrated and tested in both Routing and Vault. There were a couple of fixes regarding rate-limiter. In this commit1, we make sure to send enough information to identify and clear the message from the acknowledgement manager in case we get the rate-limit exceeded error. When this error happens, the whole message is forgotten by Routing and the upper libraries are informed, which then have the responsibility to handle it the way they want. Safe_core for instance is going to retry the operation after a wait so that it doesn’t keep getting the same error back. This PR changes GETs to PUTs (mutations) as all it is testing is whether the request reaches the correct number of nodes at the correct destination. GETs are charged for the worst case as we don’t know in advance how much traffic it is going to cause. Changing to mutations allows the test clients to invoke the RPC pertinent to the test without getting an interference from the rate-limiter.

    For vaults, we now get the group size and quorum from Routing where it can even be tweaked now that Routing has a config file. The rest of the changes2 in vaults were to integrate with the Routing changes.



  • MaidSafe Dev Update - Authenticator & API, SAFE Client Libs & More

    Last week, we released Test 18. The focus during this week has been on getting all the issues that need to get completed for Alpha 2 (including those reported by the community) added to JIRA and making sure there are no loose ends.

    With Test 17, we had the rate limiter quite relaxed and a lot of traffic was allowed. In Test 18, it is kind of the opposite since there are many more reattempts and retries that need to be happening from the clients. We are now working on striking a good balance between these two extremes and we’ll be updating the rate limiter to improve its performance and give a better user experience in terms of data traffic. With these changes, a lot more future requests should go through in the first attempt.

    Advisory and Endorsement Requests

    Members of the team, especially David, are getting quite a lot of direct approaches, either through lengthy DMs and LinkedIn messages, from people looking for advice on other projects, or to endorse ICOs. While we would like to be in a position to help, our focus on network development does not allow us time to read these messages in many cases, let alone respond. Please do not be offended by this, it doesn’t mean that we don’t see an importance in what you’re doing, we are simply very busy and your time is likely to be better spent engaging with other potential advisors.

    SAFE Authenticator & API

    Issues that were raised in Test 18 have been gathered and organised in JIRA. The new IPC for sharing MutableData is being integrated this week. Integrating the IPC functions in safe_app_nodejs and authenticator is almost completed. This feature has been the priority for us this week and @bochaco@shankar are getting this integrated and tested end-to-end with the applications.

    We have also fixed a few issues this week. The native modules were not being loaded by safe_browser on a few Windows machines which did not have the 2015 Visual C++ redistributable. When the user tries to log in, the authenticator kept spinning and it was misleading as if it was not connecting to the network. We are now handling this case and the proper error message is presented. safe_browser issue #99 is also resolved.

    When the browser is trying to load a page even if the requested page/resource is not available, the loader in the browser keeps on spinning. This issue is now resolved in the master branch of beaker-plugin-safe-app.

    Based on feedback from the dev community, the getHomeContainer API is being renamed to getOwnContainer.
    This API is used to fetch the application’s own/root container. The DOM
    APIs have also been updated with this change. Please do keep a note of
    this change in case you are trying to build the browser or your apps
    with the latest code in the master branch of safe_app_nodejs and
    beaker-plugin-safe-app. Also, @hunterlestersafe_browser issue #1003 soon.


    The Web Hosting Manager app updates the progress bar only after the file is uploaded to the network. If a big file is uploaded, the progress bar takes a long time to update, leaving the user guessing whether the application is hung or still working. This will need to use the NFS API to write as a stream instead of passing the entire content at once. safe_app_nodejs already exposes APIs for the same. The Web Hosting Manager app must be refactored to use the API to write as streams and update the progress bar as chunks of data are being uploaded. @hunterlester should be able to resolve this issue this week.

    @joshuefpull request for storing the history and bookmarks in the network. The feature looks
    stable after a couple of testing iterations and we’re expecting it to be
    merged soon. @joshuef
    is looking into the DOM APIs to enable returning the error code along
    with the error message. Right now, only the error message is returned as
    beaker browser does not allow the APIs to return complex objects. @joshuef is trying to implement this feature for returning errors.


    SAFE Client Libs

    The back-end team has been working hard on fixing bugs reported by
    the front-end team and the community, and improving the features that
    were released last week. We’ve started with refactoring the account
    packet structure and improving safe_core, the essential
    parts of the SAFE Client Libs. The account packet is the data structure
    that holds the user account details, keys, and some extra information,
    such as pointers to the root directories - and that is the part that we
    wanted to improve because it remained almost intact since the SAFE
    Launcher days, while now we have a much more complicated design, with
    apps and the Authenticator. So now we’ve got rid of unnecessary
    indirections (the pointer to the access container is now stored as a
    part of the account packet) and inefficiencies (the root directory has
    been removed and the containers info is now stored directly in the
    access container). Along the way, we’ve reduced the number of network
    requests and parallelised some operations (which previously introduced
    useless synchronisation delays), so overall these changes should make
    the UI more snappy. They’re mostly completed and likely to be merged
    before the end of this week. For more details, you can see this JIRA task

    In parallel, @canndrew@adam have been working on improving the new API for sharing Mutable Data. The basic implementation has been completed and merged and now we’re working on implementing useful comments and suggestions from @bochaco
    and the front-end team. First, we’ve decided to clearly define the key
    for the metadata entry of shared Mutable Data objects because allowing
    an app to provide its own metadata key could lead to security issues.
    For example, if a password manager app sets the metadata entry to say
    something like “this data must not be shared with other apps”, another
    malicious app requesting shared access could point to some other entry
    to be looked up for metadata, effectively bypassing the caution message
    and making a user confused about the intent of the sharing request.
    Second, the permissions part of the request has been simplified: now we
    assume that an app wants to request only Allow permissions, delegating more fine-grained permission settings (involving blacklists and Deny) to apps having the ManagePermissions privilege. All these changes go in this pull request and we’re currently reviewing them and perfecting the details.


    Next, we’ll be focusing on optimising apps revocation. We’ve started with this pull request,
    which fixes some flaws and edge cases, such as attempts to authorise an
    application during an ongoing app revocation. This is not allowed now,
    as authorisation might interfere with re-encryption of revoked
    containers. Another issue could happen if an app tried to update or
    insert new data into a container which failed to be re-encrypted - in
    some circumstances it used an incorrect set of keys and mangled the
    data. Tomorrow we’ll continue addressing similar minor (but crucial)
    problems.

    On top of that, there was a lot of smaller improvements. @marcinconfirming and fixing bugs
    found by the front-end team, such as a case when subsequent app
    re-authorisation didn’t result in the creation of app’s own container
    (if it wasn’t created on the first request). @marcinmore NFS tests and verified that a user can still log into his account and browse the network even if he’s exhausted his account balance. He’s also removed a legacy module, public_id, which remained from the SAFE Launcher and wasn’t actually used by the front-end. @canndrewupdated the Routing dependency
    to the latest version, removing the necessity of retrying requests if a
    request rate limit has been exceeded - now that this is the Routing’s
    responsibility, we’ve considerably simplified this part of the code in safe_core. Finally, @adamhas implemented
    an environment variable switch that disables the on-disk vault storage
    for tests. This change should speed up both front-end and back-end tests a bit.

    Routing & Vault

    As stated in the introduction, the rate-limiter for Test 18 is stricter compared to the previous test network. Message resend logic has also not been too efficient as Routing breaks the user messages from upper libraries into user message parts, each of which can be a maximum of 20 KiB. So if there was a user message which was 200 KiB, Routing would have broken it into 10 chunks of 20 KiB each. If a few of them were rejected, we’d currently be sending all parts again from the Client libs. The user library would then have to resend the entire message again after a pre-defined waiting period. To optimise this, we’ve modified Routing to handle the message resend at that layer directly. With the new changes, Routing will now selectively re-transmit the user message parts for which the error occurred. This encapsulates the message resend logic away from the upper libs. Routing also optimises route traffic by allowing the upper libs to indicate a maximum wait period after which message relay via further routes will not be attempted. This not only applies to rate limiter errors but also other parts such as the Ack-Manager.

    There have been a few patches that have been brought in to master to address certain bootstrap bugs from Test 18 and the Rate limiter has also been refined this last week to allow clients to scale their data traffic capacity based on concurrent client connections active at the proxy at a given time. The PR addressing the same is currently waiting to be reviewed and merged2. Rate limiter throughput calculations have also been updated to reflect the actual data throughput relayed by the proxy nodes to the clients.





  • MaidSafe Dev Update - SAFE Authenticator & API , Routing & Vault & More




    Meetups

    You may have noticed from the forum that the SAFE Network San Francisco meetup9@whiteoutmashups is helping this effort along with @hunterlester426, a tuition free university. @whiteoutmashupsthis topic5.

    Being at the epicentre of the software world and with a huge number of developers, San Francisco is one of many areas where we would like to see more engagement, and these events represent some early steps as we start a more focussed effort on spreading the word about the SAFE Network. We are now working to maximise the effectiveness of these events and have incorporated both internal feedback and feedback from members of the community. Over the next few events, we’ll be improving the way in which the network is presented, as well as improving the AV so that the content can be made available for those unable to make it to events.

    This week the team have and continue to work through a number of JIRA tasks that either fix issues raised by the community in previous tests, or that add features. This might potentially continue into the following week too. Lots of details about that below. Backend code is more or less at a frozen state in terms of required features for Alpha 2 (Authenticator) and frontend doesn’t look too far away from the same too.

    We have also spent more time planning development in recent days to spread some of the engineering responsibilities around the team as we have a couple of guys either out at the moment (Adam is on vacation and Bart is relocating to Hong Kong for a few months) or, as in @Krishna_Kumar’s case, preparing to get married.

    Congratulations from all the team here Krishna, we wish you and your soon to be wife (Rennee) a long and very happy future together, although she is potentially in for a tough time as she competes for attention with your laptop which may have to be surgically removed for the big day

    Community-driven content Continuing in the marketing vein, we would like to point out for those who have missed it, the effort being put in by a few long-standing members of the community including @polpolrene, @Krekc, @JPL and others who are starting a collaborative effort to pull together a new SAFE Network white paper/primer that brings together the most up to date content in one place. This comes on the back of a survey conducted by @JPL that gauges the SAFE communities attitudes toward the way the existing information is presented.

    Marketing

    David also took to his own personal blog a couple of times during this past week to write about The Impossible Network and to clarify the differences between the SAFE Network and alternative data/storage networks that incorporate blockchain technology. The fact that the SAFE Network completely removes intermediaries and humans from the management of our data through the network’s algorithms is a huge differentiator and the benefits of that is something we will be focussing more on in future marketing communications. We are also now starting to recruit 2 digital marketers to be based at our HQ in Scotland to help promote the network and drive engagement with the technology and the company.

    SAFE Authenticator & API

    The focus for this week has been to complete the tasks that were added to JIRA. The new IPC for requesting access to arbitrary MutableData objects has been integrated with the authenticator and also with both example applications (Web Hosting Manager and SAFE Mail Tutorial). We have tested the initial integration with the apps and authenticator. An application can set the Metadata for the MutableData in case it wants to share it with other applications as explained in this comment. We don’t force the devs to set the metadata. If the app is not setting the metadata and another application wants to access the same, then there is no useful information to be presented to the user for aiding the decision making on whether to grant access or not. Hence, we decided to provide an option to look up for the applications that have been granted access for that MutableData. With this feature, the user can see the list of applications that have been granted access before and then decide on whether or not to grant access to the requesting application. The API to get the list of apps that are granted permission for the MutableData is being actively developed by the core team and we will integrate the same with the authenticator.

    Another major improvement on the browser this week was to enable the error codes being passed back from the APIs on failure cases. We were not able to pass complex objects across the DOM APIs and thus we were only passing the error message previously. @joshuef has resolved this issue and now we can pass the error codes back to the DOM API layer.

    The API changes in safe_app_nodejs related to the new IPC for MutableData are also exposed from the DOM APIs. The DOM APIs are now up to date and in sync with the Node.js APIs.

    The upload experience of files in the Web Hosting Manager app is improved. The progress bar used to update only when the file upload was completed and when a large file was uploaded, the progress bar would stay at zero percentage until the file is completely uploaded. The user gets an impression that the app has hung, while it was still uploading the file behind the scenes. Now we have fixed it by using the NFS API to write in smaller chunks to the network and the progress bar reflects the status immediately.

    We have made good progress this week and the pending task list looks much smaller compared to where we were last week

    SAFE Client Libs

    We’ve been continuing to work on tasks that were mentioned in last week’s update particularly improvements on the authentication side in the safe_core part of the libraries. We have finished with the refactoring and optimisation of the account packet structure

    and the account registration flow, which is now only the Authenticator’s responsibility (safe_core
    is concerned only about filling and putting the account packet to the
    network). These changes helped us to simplify and decouple multiple
    modules in the libraries


    @marcin is continuing to work on improving the new ShareMData IPC request, basing on the changes by

    @adamhave been merged
    last week. After some discussions with the front-end team, we have
    concluded that the mutable data sharing flow needs to be changed a bit:
    previously, we didn’t support sharing of Mutable Data objects without
    metadata because it wasn’t clear what we should show in the UI in this
    case - a user couldn’t tell what the exact purpose of the requested
    Mutable Data is just by looking at its name and a type tag. Now, we
    decided to implement a new function in the Authenticator API to return a
    list of all apps that have access to a certain Mutable Data - this way a
    user can infer its purpose by looking at the list of apps that have
    permissions to access it. It’s outlined in more detail in the JIRA task.

    Routing & Vault

    The final PR for rate limiter6z which brings in the feature of soft capacity is now merged. We patched a bug in Routing which showed up in the soak tests after many iterations. It’s when two nodes want to connect to each other, prepare connection info request and send it to each other but are blocked from connecting to each other by the mock-crust test. The code in peer manager prepares connection info as a response to both connection info request and connection info response. Thus after the nodes realised that they could not direct connect and started to tunnel to each other which also failed for the first tunnel selected as this was also a part of the test’s simulation logic, one of the nodes got a connection info response from the other. This made its peer manager change state from searching-for-tunnel to preparing-connection-info and hand it over to node.rs. Unfortunately, this operation in node.rs was a no-op when called from handle_connection_info_response. Thus there was a disconnect between peer manager, which thought Routing was asking Crust to prepare connection info, and node.rs which ignored this and never contacted Crust. To fix this, peer manager is now patched to not change its state while searching for tunnel if it gets a connection info response. Eventually, other potential tunnel nodes would be tried and the connection would succeed. This fix is now merged. Soak tests with the latest changeset have been running for a few days now and are looking good.

    A Vault PR to integrate with the latest Routing is now up and waiting to be reviewed. It has the necessary refactor mandated by changes in Routing which no longer raises proxy rate limit exceeded errors (as it deals with them internally), the details of which were stated in the previous dev update.



  • Maidsafe - Safe Tube release

    safetube

    I just finished the last bit of code on Safe Tube, github To To access the website go to safe://safetube.jam 

    and paste the code from here
    into the browser developer’s console (right-click on website, click on
    inspect element and click on the console tab) or you can access it using
    a local webserver. The videos have to be loaded instead of streamed so
    it will take some time for the website to fully load. Feel free to give
    some spare maidsafecoins to this address:
    1Dksk7aA6PPpttznZi73d8xzsw4ERHSTLc

    Also big thanks @bochaco and @hunterlester for their help here

    If you have any issues or requested features please leave them here.

    This is part of Joseph's safe websites project. I will now release a seperate topic for each completed project as it will be easier to keep track of bugs and requested features.




  • Guys, I'm too old and lazy to figure it out. For example, there http://www.fintech.finance/01-... is a survey for investors - it's better to burn tokens after buyback or release them back. Which way is better?



  • MaidSafe Dev Update - SAFE Authenticator & API,Client Libs & More

    Today, we started testing Test 19 internally. As mentioned last week, we are continuing to work through JIRA tasks and fixing issues that need to be completed for Alpha 2. The network itself is looking good in internal tests and we don’t expect to make major changes to Routing & Vault before Alpha 2. However, there are still a few front-end issues we need to fix before releasing Test 19. Here are some of the issues we intend to address in the next few days:

    • MAID-2288: Uploading a directory with many files fails in Web Hosting Manager
    • MAID-2289: Permissions list in Authenticator always shows a full list of permissions
    • MAID-2270: Update Web API playground tool with changes in containers permissions API
    • MAID-2290: Show empty log page / log size
    • As well as other minor issues and UI changes

    We initially hoped to get this testnet out tonight, but we feel these pending issues might affect general UX quite a bit. We’re hoping to have these resolved quite soon and get the binaries out to you to test these new features. It is fantastic mind you, to not be waiting on backend, vaults and routing etc. as these bugs are annoying but not keeping us awake at night. We do feel the pain of delays, probably more than most and we are desperate for the energy test network releases give us all. While backend libs are stable and Alpha 3 work continuing from the guys there, Alpha 2 and these pending front-end tasks remain our highest priority to complete as soon as possible to get the new test network going to keep progressing with the roadmap.

    Recruitment Continues

    As discussed in previous updates we are growing the (local) team across areas including engineering and marketing. We are also now looking to recruit a Help Desk Manager and a Release Cycle Manager to aid with the ongoing rollout process.

    The Help Desk Manager will plan, set up and run a help desk (using JIRA) as the network advances through the up and coming test iterations toward beta and beyond. They will also record relevant information from test networks and compile update reports, as well as identify trends in support requests and perform root cause analysis.

    The Release Cycle Manager will be multifaceted and cover quite a lot of areas including a combination of: thoroughly testing all new releases, ensuring that all risks, impacts and dependencies are all clearly understood and communicated. They will also write, maintain and manage all the technical documentation, something that we all appreciate needs worked on. We also envision that the successful candidate would get involved in helping to showcase and demonstrate new feature releases to all stakeholders.

    What we are looking for with both roles is a very proactive approach to helping all users of SAFE software and this is another important step in positioning the company to be more outward facing and commercially minded, and to garner ever more feedback from users with a view to improving our future releases.

    We initially hoped to get this testnet out tonight, but we feel these pending issues might affect general UX quite a bit. We’re hoping to have these resolved quite soon and get the binaries out to you to test these new features. It is fantastic mind you, to not be waiting on backend, vaults and routing etc. as these bugs are annoying but not keeping us awake at night. We do feel the pain of delays, probably more than most and we are desperate for the energy test network releases give us all. While backend libs are stable and Alpha 3 work continuing from the guys there, Alpha 2 and these pending front-end tasks remain our highest priority to complete as soon as possible to get the new test network going to keep progressing with the roadmap.

    Recruitment Continues

    As discussed in previous updates we are growing the (local) team across areas including engineering and marketing. We are also now looking to recruit a Help Desk Manager and a Release Cycle Manager to aid with the ongoing rollout process.

    The Help Desk Manager will plan, set up and run a help desk (using JIRA) as the network advances through the up and coming test iterations toward beta and beyond. They will also record relevant information from test networks and compile update reports, as well as identify trends in support requests and perform root cause analysis.

    The Release Cycle Manager will be multifaceted and cover quite a lot of areas including a combination of: thoroughly testing all new releases, ensuring that all risks, impacts and dependencies are all clearly understood and communicated. They will also write, maintain and manage all the technical documentation, something that we all appreciate needs worked on. We also envision that the successful candidate would get involved in helping to showcase and demonstrate new feature releases to all stakeholders.

    What we are looking for with both roles is a very proactive approach to helping all users of SAFE software and this is another important step in positioning the company to be more outward facing and commercially minded, and to garner ever more feedback from users with a view to improving our future releases.

    SAFE Authenticator & API

    The main focus and effort during the last week on the front-end side was finalising the integration with some changes required in the safe_client_libs and safe_app_nodejs APIs to enable the sharing of MutableData which impacted several artifacts including the UIs and UX of the applications.

    As explained in the previous dev update, both the Web Hosting Manager and email app now support sharing the same public ID for creating/adding email and web services, regardless of which of them created the public ID. This implies an authorisation request sent for the user to allow/deny sharing the MutableData behind the services’ container that is associated with that public ID. Internal integration tests of this new flow in Test 19 are going as expected and no issues have been discovered so far. The authorisation pop-up in this scenario displays the MutableData’s metadata to the user, but since this information is not enforced to be set at creation time, it can be empty. Therefore, in order to help the user make a more conscious decision at the moment of authorising a MutableData to be shared, the Authenticator has been enhanced to not only display the MutableData’s metadata information, but also to show the list of applications that are already authorised to access it, which includes the application which originally created it. The tests so far are showing that this functionality is also working as expected.

    In parallel to the above, safe_app_nodejs and authenticator were adapted to accommodate to enhancements made in the way that the containers and MutableData permissions are retrieved through the APIs, making it more homogenous. One of this enhancements is to be able to retrieve not only the list of containers that the application has access to, but also to get the set of permissions the application was granted for each of the containers. This forced us to rename one of the APIs exposed by safe_app_nodejs and the DOM API, from getContainersNames to getContainersPermissions, to make it coherent with its semantic.

    One of the other goals for the last few days was to try to work on some minor issues reported by the community either on GitHub or on the forums, and several of them were fixed or verified to be already solved, while others are being actively reviewed at the moment, while we also try to provide as much of a help as possible to the developers asking questions.

    As mentioned in the introduction above, the main focus for the next few days is to solve a few issues that we prefer to close before sharing the new binaries with the community, as well as perform more tests to make sure that all the pieces are working fine together; in the front-end there are several integration points and covering all flows is not a trivial task.

    SAFE Client Libs

    (cargo-release) safe_core- version 0.26

    Downlaod:

    Routing, Vault & Crust

    (cargo-release)  version 0.17.0

    Download:



  • MaidSafe Dev Update - SAFE Browser, Web Hosting Manager, SAFE Authenticator & API & More

    Today, we are releasing updated binaries for SAFE Browser and Web Hosting Manager (to fix a few minor issues):

    Download SAFE Browser

    Download Web Hosting Manager

    These binaries will connect to Test 19  (the test network we released last week), so you shouldn’t need to create a new account. See the Test 19 topic for more information.


    As you can imagine there is a lot happening in the background with Alpha 2 and Alpha 3 release planning. We have multiple teams all working on the various components now to ensure we deliver these next two releases promptly and professionally. You will have seen recent recruitment drives to aid us in these next few steps.

    Alpha 2 is not far away as we all know, but a timely release is imperative now and must tie up with other parts of the project such as PR, marketing and business relationship building. The Alpha 3 meetings this week have been very productive and allowed us to find some really pleasing numbers regarding network or overhead cost of some network actions. All in all, this has been a very good week.

    SAFE Authenticator & API

    Since the release of the front-end applications with Test 19, no major issues have been reported, neither internally or from the community. This is a good sign that the applications are quite stable and it is encouraging us to keep working on the minor issues we have in the backlog. We really appreciate the feedback and also the issues that the community has been reporting, and it’s very important and very helpful to have all issues reported, even if they are minor, in order to enhance the applications and make them more stable and usable.

    The following issues/tasks were resolved during the last few days:


    • MAID-23272: minor refactor to safe_app_plugin to remove the queue for auth requests as the Authenticator already provides one.
    • MAID-2307: solved the issue that causes a crash in the browser when browsing a deleted service
    • MAID-2306: uploading empty directories is now correctly handled.

    The issue with saving the state of the browser on the network (MAID-2318) is fixed but still undergoing internal testing. It should be solved in the next version of the browser.

    As anticipated in last week’s dev update, @shankar resumed the effort to enhance the UI/UX of the Web Hosting Manager application; we hope to be able to start our internal testing within the next couple of days.

    @joshuef is now working on trying to automate our release/packaging process since we’ve been doing it manually so far, which is not only a time-consuming task but also very error prone. This will help us in the short term to be able to spend less time on this process and also allow us to be able to quickly respond if an urgent patch is required for any of the front-end applications.

    Additionally, @hunterlester is working on creating tools/mechanisms to automate our tests for the DOM API. This is also in line with our goal of being able to quickly provide patches/fixes without risking the quality of the software.

    SAFE Client Libs

    We improved the app authentication process: the response from the Authenticator now includes the full Access Container Entry (a structure that contains details about the containers the app has access to), saving the app a network request. We also simplified and optimized the API for working with MDataInfos (structures that contain the network identifier and encryption keys of a mutable data).

    MAID-23132 is nearing completion which will allow us to automatically generate C header files to interface with our FFI. As part of this task, we refactored the FFI modules of safe_core and safe_authenticator. We also caught several FFI functions that were either not marked #[no_mangle] or were missing an extern declaration.

    Routing & Vault

    We are continuing to focus our research efforts on message broadcasting strategies, or in other words - strategies for ensuring that every intended recipient of a message eventually receives it. In addition to the simulation6 we developed last week, a further investigation on what additional penalty cost our current approach incurs is being carried out. We hope this will clarify our current situation and could provide us with a clearer comparison to any future proposal. Some other metrics, such as the total amount of traffic or number of messages in various approaches, are also being calculated.

    Crust

    Work this week has focused on tracking down bugs in our uTP implementation. Currently, the SAFE Network uses TCP for all peer-to-peer communication, though there are many advantages to using a UDP-based transport protocol such as speed and easier hole-punching through home routers. Our aim is to have uTP in a state where we can deploy it alongside TCP sometime in the next couple of weeks.



  • MaidSafe Dev Update - SAFE Network goes mobile, Alpha 2 - The Authenticator & More

    Another very busy week on the development front and as you will see we
    have a few things to get through in this update, much of which will come
    as welcome news to anyone following our development progress.

    SAFE Network goes mobile

    As you will soon notice, this update comes with binaries, but
    unusually these are not only for desktop, these also include mobile
    binaries. The two mobile apps in this repository 

    a mobile Authenticator and SAFE Messages, a mobile email app, are tech previews.
    These are not meant as end user applications, this example is to show
    developers one way to integrate from these platforms. We will allocate
    time and resource to create a full developer experience on mobile via
    native SDKs, including an easy to use API and documentation. This
    preview merely showcases the network’s mobile compatibility. Currently,
    these applications are created with Xamarin

    as it allows for a simple unified codebase to test the compatibility
    concepts. We wanted to confirm the authenticator capabilities from
    mobile platforms and check out any unforeseen issues pre Alpha-2 and
    were pleasantly surprised that the native code was very close to plug
    and play for mobile platforms (a few changes were needed, we’ll probably
    cover the mobile integration and the journey of bootstrapping with the
    network from mobile platforms in a separate post to not turn this into
    an essay

    1-login3-home5-request-authorisation6-auth-request

    7-email-accounts9-single-account10-single-message11-reply

    Android binaries

    Safe Authenticator ( Andriod ) on github:

    Safe Massages ( Andriod ) github:

    Read Full Article https://safenetforum.org/t/mai...



  • Maidsafe (Maid) Release SAFE Browser v0.5.3

    Change log

    • Remove safe-app dependency
    • Check if webContents is not null after fetching current focused window - fixes #162
    • Authenticator fix for handling multiple Mutable data authorisation request

    SHA-256

    Linux
    FE4D5EE4F966095DFBE8E4E521B17B4967420309A4B86A43C4E3A0C66BF166A5
    OS X
    D01C01BF87D87700A486AADFB6F1CC4A22B6C5EB5588B431CF5E383A868D0FEB
    Win
    06AE230221AD1A38C4ABFA9C5532C770E2F12755E39747759DA0FED024CA90A4
    

    Download:https://github.com/maidsafe/sa...



  • MaidSafe Dev Update - Alpha 2, SAFE Browser v0.6.0 & More

    As you all most likely know this is no ordinary dev update. This update announces the formal launch of Alpha 2! This release is called ‘the Authenticator’ and focuses on the network access mechanism of the same name. The Authenticator makes a number of improvement over its predecessor (the Launcher) not least mobile support in the shape of Android, which we previewed last week, while iOS support is well underway and will be released in the not too distant future.

    The Authenticator, bundled with the SAFE Browser, provides account access control that is now enforced by the network as opposed to the Client. For developers, this release contains more fine-grained access control through the use of smart data types.

    We continue to support the 3 major desktop platforms and the tutorial apps for this release are the Web Hosting Manager and SAFE Mail. In addition, on Android we have the Authenticator and SAFE Messages. These are all available to install directly from our new MaidSafe.net site. As you will notice this is a release specific website and hats off to @shona who came through with the design and @shankar for the very speedy implementation. We hope you like it! As we tick of this next major milestone it would be remiss of us not to thank you all, the SAFE community, who have tested every release we have put out. Your continued feedback is hugely helpful to us. Without the insights this community provides, our releases would not be as usable or as fun to create. Our hats off to each of you from everyone at MaidSafe! This is a new network, so trying to access data from Test 19 won’t work (you will need to create a new account). We are planning to take Test 19 offline this weekend. The number of clients per account that can connect at a time has been increased to 8 clients instead of 4.

    SAFE Browser v0.6.0


    safe-browser

    Changelog

    • Storing history and bookmarks to network
    • UI improvements
    • uses authenticator plugin v0.3.0
    • uses safe-app plugin v0.3.0

    Known limitations

    • Localisation is not supported.
    • Unicode characters are not supported in public IDs, services names, or file names.
    • When there is a network latency the browser can freeze for a few moments while it’s trying to communicate with the network for different operations, like reading/saving bookmarks & history entries, loading a website, etc.

    Example applications

    Please note we are not supporting either localisation for the apps nor Unicode characters in the public IDs, service names, or file names for the web services, although they are supported in email IDs.


    Download SAFE Browser v0.6.0

    Web Hosting Manager v0.4.0

    web-hosting-manager

    Manage web hosted contents in the SAFE Network using the Web Hosting Manager. Create services/websites and upload web content that can be hosted in the network. The hosted services can be viewed using the SAFE Browser.

    When uploading files using Web Hosting Manager, the maximum file size is 20 MB per file.

    Note: To upload a website, you’ll need to upload the files and folders separately. This is because we added support for uploading subdirectories. Ideally, it would be possible to choose both files and folders at the same time, but it looks like Electron doesn’t provide the upload dialog option which will let users choose files and folders. Hence the current mashup of both options. So to upload your website, you might need to first upload the files from the root of the website by selecting all of them via the upload file feature and then choose the folders via the upload directory feature.


    Download Web Hosting Manager v0.4.0 on GitHub

    SAFE Mail Tutorial v0.4.0

    safe-mail-tutorial

    Send secure messages using Messaging/Email Example app between users. Create an ID and send the messages between users from desktop and mobile.


    Download

    SAFE Web API Playground

    The SAFE Web API Playground allows any developer to learn and experiment with the DOM API in a friendly and interactive way.

    It is meant to help people aiming at creating websites/webapps that need to access the SAFE Network DOM API, which is available on the global window object in the SAFE Browser.

    Instead of having to upload your site to the network every time you want to test some code, you can simply use this tool to experiment and get a deeper understanding of how each API works.

    In order to use it, follow the instructions found in its README, and either upload it as a web app using the Web Hosting Manager application or just run it locally and access it from the SAFE Browser using the now supported localhost protocol (see above for details).

    Repository

    Clone

    Mobile examples

    Demo mobile applications that showcase some of the initial key features of the Alpha 2 SAFE Network. For now, only Android is supported in devices. These apps can be used in conjunction with the Alpha 2 desktop apps; the SAFE Browser and the Email/Messaging App.

    SAFE Authenticator

    3-home
    3-home.png1440x2560 76.2 KB

    Download


    • Download 10-single-message
    • 10-single-message.png1440x2560 67.6 KB

      Download

      • Download #support category of this forum.

        For more technical questions (e.g. about the SAFE APIs), please use the #supportSAFE Dev Forum.

        Where should I report issues?

        If you know which component a bug is associated with, feel free to report it on GitHub in the corresponding repo.

        • Report issues with Report issues with Report issues with Report issues with Report issues with Report issues with Report issues with Report issues with

          SAFE Authenticator & API

          The biggest change this week comes to the Web Hosting Manager app, which has been updated and improved by @shankar@shona with a new UI and UX. We’ve included some fixes based on the feedback from the previous release, as well as on findings and feedback from our internal testings performed in the last few days to make sure the new version is as stable as the previous one. We believe this new version is not only nicer but also easier for people who never used the tool or the SAFE Network before, since it provides hints of usage in different parts of the tool.

          @hunterlesterstarter pack13 to help developers get started with building SAFE web apps :slightly_smiling_face:

          We’ve also set up the foundations of a test suite for the browser DOM APIs, which should help prevent problems as things evolve there. @joshuef has been trying to debug and improve the reliability of the authenticator test suite and npm package issues.

          Some minor internal changes which don’t affect the API were made to the safe_app_nodejs package in order to make use of the latest version of the safe_client_libs which includes some minor fixes and several internal enhancements as explained in the next section below.

          During the next few days, we will continue our efforts on automation of tests and artifacts packaging, and we will start reviewing and discussing tasks we have in our backlog to prioritise them and think of enhancements needed for next releases.

          SAFE Client Libs

          Yesterday, we released new versions of SAFE Client Libs: safe_app & safe_authenticator have been updated to 0.4.0, safe_core to 0.27.0, and ffi_utils to 0.3.0. These versions are published on crates.io now and they contain all recent changes in SAFE Client Libs.

          We wrote a couple of NFS tests this week for reading and writing to files in chunks for MAID-2352MAID-2357. We also documented FFI callback parameters throughout our API, making it clear what each variable is, in MAID-2322Issue 445701 to get resolved on the Rust side, but each callback parameter in safe_authenticator and safe_app has also been documented in doc comments. In addition, @marcinC header generation1 and writing missing test cases1) have been reviewed and merged.

          There were many improvements and refactorings that are not visible on the surface (i.e., the API hasn’t been changed). With the help of the front-end team, we also discovered and fixed a bug in the way we handle the Object Cache, the storage tied to the App object that maps object handles to the actual objects. Currently, the Object Cache is implemented as an LRU cache, meaning the stale objects get freed automatically, and it was implemented in the same way in the SAFE Launcher and REST API times. This approach simplified memory management a lot, but nowadays we have switched from the REST API to the FFI API, and memory management became more straightforward. But the Object Cache remains, and the fact that it is limited to 100 entries for each object type can result in some obscure bugs, e.g. when you try to create more than 100 Mutable Data objects at once. We fixed this by raising that limit to 1000 entries temporarily, and we’ll be looking into resolving this problem in a more general way by switching from the LRU cache to a plain HashMap, thus making it the front-end apps’ responsibility to free stale objects and prevent memory leaks (and the Node.js library already does that, so it should not affect it). Also, @adam found that some FFI functions miss the catch_unwind wrapper, which could have resulted in Rust panics not caught and propagated to FFI - or, in simple words, in crashing apps and the browser. This issue is fixed by this PR.

          We improved error reporting for account creation, to help the user better determine the actions to take to recover from the error. We also fixed a couple of minor bugs that manifested themselves only on the nightly compiler. This compiler is used by docs.rs1 to generate the API docs, so by fixing those bugs, the docs are now being generated again.

          Routing, Vault & Crust

          Work this week has again been focused primarily on the design of the message flow. Several ideas have been explored and a lot of effort has gone into finding flaws in these and looking for ones with the potential for later optimisations. A long-standing idea to use a form of Information Dispersal Algorithm to increase the efficiency of message-passing (and perhaps also data-storage) is being explored further. There are still some unresolved issues around this proposal, but we’re hopeful that these can be addressed since the apparent benefits of this approach would be fairly significant.

          Over the last couple of weeks, we’ve been improving our implementation of uTP (BitTorrent’s micro transport protocol). It’s now in a usable state and work has now begun on integrating it into Crust. At the moment, this means seeing how our existing code can be refactored as Tokio-style state machines, starting with our NAT traversal library.




  • [wanted] Developer for new SAFE project!

    https://safenetforum.org/t/wan...



  • Maidsafe - SAFE Browser v0.7.0 - Release

    A new release of the SAFE Browser (v0.7.0) compatible with Alpha-2 network has been published.

    This is the first release in which we are formally making available packages to connect to the Alpha-2 network, as well as those which connect to the MockVault. We are planning to provide both flavours from now on.

    Please note that the artifacts for Windows and OSX were not signed this time, so please use the provided SHA-256 hashes instead.

    Changelog

    • Fix the issue with favicons which are now loaded and displayed for safe:// sites.
    • Warn the user upon a network disconnection event in the browser, not only from the Authenticator page but in any open tab, and attempt to automatically reconnect every 30 secs if the user doesn’t explicitly do it.
    • Assure that when reconnecting to the network, not only the Authenticator connection is re-established but also the ones for the safe_app plugin and the browser app so the browser state can be saved on the network afterwards.
    • Functionality to run tests on the DOM API functions within the browser, and creation of a checklist document for manual testing.
    • Fix URI scheme registration for dev mode so the browser can receive authorisation requests even when launched with yarn start.
    • Minor enhancement to README for Windows build instructions.
    • Fix minor issues when fetching sites with a relative path.
    • Uses authenticator plugin v0.3.1.
    • Uses safe-app plugin v0.3.2.

    Download

    Download SAFE Browser v0.7.0 on GitHub

    Where should I report issues?

    Feel free to report it on GitHub in the SAFE Browser repository



  • Maidsafe Dev Update - SAFE Authenticator & API,SAFE Client Libs & Routing, Vault & Crust

    Today, we are releasing an editable comments plugin It’s similar to the comments plugin we released a few weeks ago (see this dev update3), except that it allows users to edit their own comments and to see the version history of each comment.

    edit_popup

    after_edit

    show_version_diff

    This JavaScript plugin can be integrated into any safe:// web page by following the instructions here. We’ve also set up an example at safe://blog.safe-examples7 (on Alpha 2) for community members who want to try a hosted version of the editable comments plugin. To host your own version of this example, you can download example-blog.zip from here8 and upload it to the SAFE Network (Alpha 2) using the Web Hosting Manager application.

    SAFE Authenticator & API

    During the last few days, we’ve been progressing well on making the necessary changes in safe_app_nodejs to be able to upgrade to the latest code from safe_client_libs. There is only one minor issue detected which is being investigated at the moment. The corresponding changes in the authenticator plugin were also made to make use of the latest authenticator code from safe_client_libs.

    We started working on the bindings to expose new functions from the safe_app_nodejs API which are provided by the new safe_client_libs, like the functions to manage signing keys (generating them, signing and verifying signatures), the ability to query for containers permissions granted just using the authorisation URI without the need to connect beforehand, and the ability to access the account’s information which contains the number of mutations done and available.

    After finishing with all changes in the safe_app_nodejs API, we plan to make the necessary changes to our example applications so they can make use of the latest version of safe_app_nodejs and safe_client_libs. All these changes will be compatible with the Alpha 2 network and stored data, therefore during the integration tests, backward compatibility tests will also need to be performed.

    The scripts to automate the tasks of packaging and publishing both of the SAFE Browser’s plugins on the S3 servers have been put in place, and they are ready to be used in our next release. They are automatically triggered when we push a new tag to each of the plugins’ repositories. Analogously, a script to automatically create the safe-node-app package and publish it on npmjs.com is in place and ready to be used in our next release.

    @srini has created an editable-comments-plugin6 example to demonstrate how versioned data can be handled using the APIs. @shankar has started to work with @shona in planning the developer website.

    @rachit and @krishna_kumar have set up the Java repository for safe-app3 using Gradle. Composite Build Setup of Gradle is used to set up the project. Since this feature of Gradle is still in incubating stage of development, only the latest versions of the IDEs support the same. Android Studio version 3.0 and IntelliJ 2017.2 support composite builds. This setup helps us to have one shared codebase for both Android and desktop library files. The repository is set up to get JAR files of all combinations (macOS/Windows/Linux, Mock/NonMock, x86/x64) for desktops and mock/non-mock versions for Android.

    With @joy (Janmejoy) joining the team this week we will be able to focus on Java and C# in parallel.

    @joshuef has been focusing on the custom browser and has been progressing well. The POC consists of a basic HTML browser with a simple UI. The new browser has a comprehensive test suite and more robust logging, both of which should help speed up development of new features. Now we’re progressing with a much tighter integration of SAFE protocols, which should also lend itself to a more streamlined development process going forward.

    SAFE Client Libs

    During this week, we’ve been working on catching and fixing bugs, including the ones reported by the community. The first bug concerns the apps revocation feature: in some rare circumstances, users could be locked out of the ability to revoke any apps. Thanks to the very detailed report by community member @JPL, we’ve been able to pinpoint the cause of it quickly, and it turned out that our revocation code didn’t account for the way entries are stored in the apps’ containers by some of the community apps. So that when the re-encryption process kicked in, it tried to decrypt those entries, and because some of them were unencrypted, it failed, effectively failing the entire revocation process. Now that we have errors recovery, this re-encryption was attempted repeatedly, resulting in continuously failing revocation attempts. This bug is fixed and merged now and should be integrated into the SAFE Browser soon.

    We also fixed a bug with the is_mock_build() function in safe_core not working correctly. The cause was simple (invalid syntax which the compiler didn’t catch) and the fix was simpler.

    Another bug was in the system_uri library and was present only on the macOS platform. The underlying cause of it turned out to be quirky, so it took some time to investigate it: the open function exported by the library didn’t work correctly and resulted in a segmentation fault, which basically means that system_uri wasn’t usable on macOS and because of that the front-end team had to invent some temporary workarounds. Apparently, the name of the open function itself clashed with some system functions on Mac, so just renaming that function did the trick and the bug is now fixed.

    In the background, we’re also continuing our work on the bindings generator. It is progressing quite well and we’re almost done with the Java part. The major remaining thing to be done is the JNI glue code generator. It required some time to research and decide which way forward is the best as we have two options: either generate C++ JNI binding code (which is a standard practice in the Java world) or write this code in Rust (which is, by and large, experimental for now). Both of these options have their pros and cons, but ultimately we decided that keeping our language stack small and focussed outweighs the possible disadvantages of writing JNI code in Rust. Another benefit that we have with Rust JNI bindings is that it’s much easier to generate Rust code programmatically because we have full access to the relevant APIs of the Rust compiler (namely, the syntax parser and the pretty-printer). In other good news, the code generation framework is essentially complete and soon @adam will start working on the C# bindings generator. The current progress with it can be tracked in this repository

    Routing, Vault & Crust

    We’ve been pushing ahead with the Data Chains specification this week, having chosen that proposal over the Section Graph one. We’re well on the way through the mid-level design; somewhere between the high-level overview of the protocols and the low-level programming tasks which will be finally produced. So far, this effort to produce more detailed plans has resulted in several lengthy design discussions involving the full design team. The majority of this work is complete now, and we’re hopeful that in the next few days the actual tasks can be specified so that we can begin turning this proposal into code.

    The migration of Crust to the new Tokio + Futures architecture is complete. Compared to the previous iteration of Crust, the code is much smaller and more easily maintainable. This will make it much easier to implement the future improvements that are planned for Crust. Speaking of which, a first prototype of the NAT-traversal crate is now complete. Although further testing is needed before we can be confident in deploying it, it is ready to start being used in the development branch of Crust. In the coming weeks, with our new NAT-traversal and uTP crates, we should have a version of Crust which can also run entirely over UDP (in addition to TCP).



  • Maidsafe Dev Update - SAFE Authenticator & API, Client Libs , Routing & Crust

    Here are some of the major areas of focus this week:

    • safe_app_nodejs API updates
    • safe_client_libs version updates and Alpha 2 compatibility
    • Java and JNI bindings generator
    • C# bindings generator
    • Merge and split scenarios in the context of Data Chains
    • Migration of the p2p crate to a Tokio + futures based API

    SAFE Authenticator & API

    The tasks to update safe_app_nodejs to make use of the master branch of safe_client_libs have been completed and it’s ready to start the internal testing. All API changes and the new functions for sign keys have been integrated. We will now update the example applications to work with the latest version of safe_app_nodejs and also use them for testing purposes.

    The changes which affect the API are the following:

    • Permission-set objects no longer exist on the front-end, they’ll simply be passed as a clean array:

      const permissionSet = ['Insert', 'Update', 'Delete', 'ManagePermissions'];
      
    • You’ll no longer have to work with MutableData Keys and Values objects. For example, you can now simply run app.mutableData.getKeys() to return a nice array of keys associated with the respective mutable data entries.

    • See merged PR for full details, https://github.com/maidsafe/safe_app_nodejs/pull/166/files4, while we update documentation.

    The DOM API is being updated to reflect the changes made in safe_app_nodejs. The documentation of the new APIs will be updated and published as soon as this is complete. As you can imagine, some of these changes will reduce the number of handles the DOM API needs to expose, and that should simplify the web apps code to a certain degree.

    The test suites for both the DOM API and safe_app_nodejs are in the process of being updated to reflect recent safe_client_libs changes, for readability, and to perhaps act as another source of practical documentation. We are also working on adding tests which can verify that invalid input parameters are handled correctly.

    We just started with some refactoring in the safe_app_nodejs code with the intention of decoupling the NodeFFI binding functions, which take care of the interactions with the safe_client_libs, from the functions that are exposed as the API of the package. This should allow us to then be able to use different types of bindings mechanisms as we find it necessary moving forward, e.g. we could create different versions of the library which interact with the safe_client_libs using the Rust FFI, WebAssembly, or any other possible way.

    The safe_app_java repository has been updated to use multi-project setup and CI scripts have been integrated. We’re waiting for the JNI bindings to be ready for integration. Meanwhile, @joy is looking into methods of distribution. @shankar is iterating the dev website prototype based on the initial feedback that was passed by the internal team. We will share the prototype once the final version is ready.

    A test suite for the existing APIs was integrated for the safe_app_csharp repository. Only a few functions that were needed for the messaging app have been exposed as of now. @rachit and @krishna_kumar will start implementing all the APIs and corresponding test cases to make it feature complete.

    The custom browser development is continuing well. The prototype protocol handling was functioning well, but it required running the old SAFE Browser for authentication. Now we’re moving on to integrating the authenticator to remove this dependency.

    SAFE Client Libs

    The highlight of this week is our work towards getting an Alpha 2-compatible release of a new version of the SAFE Client Libs API. Currently, we’re trying to check that all recent changes in the master branch of SAFE Client Libs are working nicely with the existing network data. Besides manual testing, we intend to add some automated integration tests that will try to connect to the real network and use an existing account to do basic tasks: this way, we can be sure that developers can use the new API to build new apps and upgrade the existing ones without resorting to mock-routing or waiting for a full network reset. Once we’re confident about it, we’re going to remove the alpha-2 branch and make master the only upstream version.

    As we move forward, we’ll be dedicating more time to the problem of binary data compatibility. We’re starting with small steps: @marcin has been developing an update policy for our dependencies, some of which can change their binary encoding from version to version. We’ve already stumbled upon this with our compression library, brotli2, and the same might happen with e.g. our serialisation library (serde2). As updates to binary encoding are breaking changes, we decided that we’re going to freeze versions of such critical libraries. In addition to that, we’ll be more careful with future changes and general dependencies updates. The outline of this policy can be found in this pull request2, which will be revised and adopted soon.

    In the meantime, we’re continuing with the bindings generator, which is progressing very well, but still requires more time to figure out the small details and catch bugs. While @nbaksalyar is wrapping up JNI and Java bindings, @adam has started on the C# bindings generator. It’s shaping up nicely and some preliminary results are available in this repository2. The major remaining thing to be done for both languages is testing and proper callbacks support. Once we’re done with that, with the help of the front-end team we’re going to start integrating automatically generated bindings with the Android and Xamarin projects.

    Routing & Crust

    The Routing team has been focused on working out the details of how merges and splits can be effectively handled in the current design of Data Chains. We are considering various scenarios and different flows, trying to find a suitable approach that plays well with eventual consistency patterns. This is ongoing and we’re hoping to get things firmed up for merge/split handling soon.

    The task of figuring this out requires a thorough understanding of the network fundamentals and Data Chains features. We’re also looking to factor in quite a few drastic scenarios such as high node loss in short periods, restarting the network with the ability to republish the data that has been stored on it… It is important to make sure that these features will be possible to implement on top of Data Chains, and many details are quite tricky to get right and ensure they function together without affecting a different part of the system as a side-effect.

    We’ve been planning to replace Crust NAT traversal code with a dedicated crate - p2p6. But before that, it had to be ported to a Tokio + futures based API. And that finally happened and got merged into the master branch. This concludes our 2 week long work - that was a complete rewrite from mio based code. Now we’re able to integrate the p2p crate into Crust and the work has already been started. The end result will be more reliable NAT traversal with a reduced Crust codebase. Since p2p implements both UDP and TCP NAT traversal and µTP is ready, UDP support for Crust is not that far away.



  • Maidsafe - Project Decorum Wallet Release

    We are happy to announce the release of our first experimental app module, the Project Decorum Wallet!

    Features The wallet implements our novel token design that can be considered a hybrid between the typical SAFE MutableData token and the blockchain tokens that are common in the crypto-currency scene:

    • Bitcoin-style divisibility, up to 8 decimals of precision without any additional performance cost from using those decimals
    • Records the history of all transactions in a tree structure, while only revealing transactions on a need-to-know basis
    • Transaction validation only requires validating the direct ancestors of received transactions
    • A transaction “explorer” similar to classic blockchain explorers that allows you to inspect the history of your transactions all the way to the genesis transaction
    • Allows you to easily create you own token (fixed supply only for now)
    • Supports sending tokens directly to someone’s SAFE name (public ID)
    • Allows you to create multiple named wallets and SAFE names (public ID’s)

    For more technical details, check out this presentation10 of the SAFE Dev 2016 Amsterdam meetup.

    Usage The official SAFE Browser 0.7 is required, you can download it here21.

    After installation, create a SAFE account if you don’t have one yet, login and then visit this SAFE location:

    safe://wallet.decorum16

    On the main page, first create a named wallet. Go to ‘Names’ in the navigation bar and register a SAFE name if you don’t have any yet. Then couple your SAFE name with your wallet on the same page. Transactions can now be sent to your wallet simply by using your SAFE name as the recipient! You may post your SAFE name(s) in this thread so we can all send each other tokens for fun.

    Motivation We strongly consider to use this token design for Project Decorum’s Clike token that can in the future be exchanged for Project Decorum Coins. Although this design also has performance trade-offs compared to regular MutableData tokens, we hope to see a net improvement for Clike’s intended use as endorsement tokens. The high degree of divisiblity with no performance cost is also desirable.

    Developing this particular part of Project Decorum’s goals first was done for several reasons. Since the design of SafeCoin (especially concerning divisiblity) is not necessarily final yet, we hope this experimental design can offer new insights and spark ideas and discussions that may be useful for SafeCoin and perhaps also for tokens of other SAFE projects.

    Another reason to develop and release this module first was because it’s narrow in scope while almost all of SAFE’s API features and a lot of cryptography had to be used. As such it functioned as the perfect crash course for developing apps on SAFE, which requires a different way of thinking than regular client-server applications on the internet. Our experience of the past months will help a lot with further development of Project Decorum’s social networking features.

    Disclaimer This is an experimental release, it shouldn’t even be considered to be at an Alpha stage. None of the tokens in the wallet represent any value. If you have programming skills it’s possible to break transactions if you know what to do. This is because there’s still one final feature missing from the current SAFE Browser DOM API that we need to make transactions truly immutable.



  • Maidsafe Dev Update - SAFE Authenticator & API & More

    We have progress to report in a number of areas this week with the front-end team delivering a few upgrades. As you will see, the SAFE Browser v0.8.0 is being tested internally, along with new versions of the Web Hosting Manager and the Email app having incorporated the latest changes from safe_app_nodejs and safe_client_libs. These will be pushed out just as soon as we have verified they are working correctly. The new custom browser Proof of Concept (PoC) is also going through internal testing and while basic seems very functional.

    The Routing design discussions continue with particular focus being given to managing section merges with the integration of Data Chains. The team continue to consider edge cases to cover as many scenarios as possible. It is much more efficient for us to plan this stage very carefully as moving to code/tasks too quickly and failing to consider a particular event will only cost us more time in the long run. Things are moving in the right direction, there is skeleton code in place as a baseline and we hope to have more progress to report soon.

    SAFE Authenticator & API

    We finished making the required changes in safe_app_nodejs to upgrade to the latest code from the safe_client_libs and system_uri libraries, and we were able to successfully package and publish v0.5.0 of the safe-node-app package on npm for the first time using the automated scripts. This new version exposes some new useful functions in its API for applications to not only generate sign keys but also sign and verify content with them.

    Both the safe-app and safe-authenticator browser plugins were also updated to work with the latest versions of safe_app_nodejs and safe_client_libs, and they were also packaged and published on our S3 servers with the new automated CI scripts.

    A new version of the current SAFE Browser, which makes use of the latest packages mentioned above, has been made available today for internal testing. Tests to make sure that current data in the Alpha 2 network is still accessible after the changes we made in our packages need to be performed. This is also the case for new versions of the Web Hosting Manager and the Email app which are also going through an internal testing phase.

    The first and early-stage version of the custom browser has also been made available today for internal testing. Even though some aspects of the UI are still very basic and functional, this first version is already allowing the user to log in to the Alpha 2 network, browse safe:// websites and authorise standalone applications with its embedded authenticator, just like our existing SAFE Browser & authenticator. We are performing tests on all platforms so that we can share it with the community ASAP.

    Some of our next tasks are related to moving forward with the automation of our release process, focusing now on automating the packaging and publishing tasks for the browser on all supported platforms, with versions for mock and non-mock routing. We are also resuming some other activities in relation to enhancing our tests’ code coverage in safe_app_nodejs as well as some internal refactoring of the package.

    The existing APIs in safe_app_csharp are being updated to work with the safe_client_libs master branch. safe_app_csharp currently supports the mobile iOS and Android platforms, we are also working on adding support for the desktop platforms which should be live soon. We are also working on expanding the test cases for the existing APIs. JIRA tasks were created for adding the missing APIs to complete the safe_app_csharp library.

    The progress in Java APIs has been more focussed on packaging and distribution practices. @joy will be using the Java bindings to start wiring up the APIs from early next week.

    SAFE Client Libs

    We’re making progress with language bindings: this week we produced first results for both C# and Java/JNI. This hits a big milestone but there are still some small things to be completed. First, JNI structures have to be mapped to our native Rust structures. Second, C# bindings need to be accommodated to Xamarin to work on mobile platforms (and on iOS especially, as this platform has some extra requirements). After we’ll be done with that, we’re going to work closely with the front-end team to improve upon the generated bindings and produce some small working examples of the API usage; we’re expecting this to happen early next week. For Java, we’re also working on merging some changes3 that we made to the JNI library to the upstream.

    In parallel, @marcin has been working on improving SAFE Client Libs’ nuts and bolts. After we made this change to the way URIs work in the inter-process communications between apps and the Authenticator:

    We stopped prepending schemes to URIs so that the SAFE Client Libs are decoupled from the URI scheme and only concern themselves with payload encoding/decoding

    there was no apparent way to provide an unregistered app ID within a request, so the front-end part of the Authenticator didn’t really know where to send the response. This has



  • MaidSafe Dev Update -  Authenticator & API , SAFE Client Libs & More

    Today, we were planning to release new versions of the SAFE Browser and the example applications which have been updated to work with the latest API changes. The changes are mostly related to the safe_app/DOM APIs and also to improving the developer experience.

    However, there is a limitation in the browser which we are working on fixing at the moment. If we open a web page, say www.example, it loads fine, however, www.example/index.html is not being loaded. We will be releasing new versions of the browser and the example applications as soon as this issue is resolved (tomorrow hopefully).

    As we mentioned in a previous update4, we would keep you up to speed with developments in our new Indian office. Circumstances have enabled us to take a bigger office for the same monthly costs within an IT-specific business park. The additional square footage will facilitate up to 30 staff as opposed to the previous 15. The one caveat is that the bigger office space requires more internal work and will therefore push out the anticipated opening of the office to late January, however, we felt that the bigger space was worth the slight delay. With the front-end team starting to make regular new releases, there is much to be excited about bringing this team closer together and growing its capability even further.

    SAFE Authenticator & API

    SAFE App Node.js2 & SAFE Web AP have been updated to adapt to recent API changes in SAFE Client Libs, summarised as follows:

    SAFE App

    Addition of following functions to app object in safe_app_nodejs and window.safeApp in web API: isNetStateInit, isNetStateConnected, isNetStateDisconnected, clearObjectCache, isMockBuild.

    Crypto

    Addition of a signing key API on app.crypto object in safe_app_nodejs and window.safeCrypto in web API: generateSignKeyPair, pubSignKeyFromRaw, secSignKeyFromRaw, generateSignKeyPairFromRaw.

    • signKeyPair.secSignKey.sign or window.safeCryptoSecSignKey.sign to sign data on the network
    • signKeyPair.pubSignKey.verify or window.safeCryptoPubSignKey.verify to verify signed data on the network
    • Note that the safeCryptoKeyPair object is replaced by safeCryptoEncKeyPair and safeCryptoSignKeyPair in safe_app_nodejs andwindow.safeCryptoKeyPair is replaced by window.safeCryptoEncKeyPair and window.safeCryptoSignKeyPair in web API.

    Mutable Data Permissions-set

    Removes MutableDataPermissionsSet and window.safeMutableDataPermissionsSet.

    Mutable Data Permissions

    forEach changed to listPermissionSets, which now returns an array of permissions represented as strings, instead of the no longer existent permissions-set object.

    Mutable Data

    • removed free function
    • removed newPermissionsSet function
    • getKeys returns array of mutable data keys
    • getValues returns array of mutable data values

    Mutable Data Keys

    Removes MutableDataKeys and window.safeMutableDataKeys.

    Mutable Data Values

    Removes MutableDataValues and window.safeMutableDataValues.

    It’s suggested to browse documentation here to learn the APIs:

    And to read the tests for each API to view examples of how each function works:

    We have an issue with the loading of certain URLs in SAFE Browser. We are working on addressing this issue. We will be releasing the browser and the example applications as soon as this issue is resolved (tomorrow hopefully).

    @joshuef has been working on the custom browser over the last couple of weeks and has made considerable progress. We are testing the initial version of the custom browser which makes use of the latest APIs and allows safe:// only browsing. The UI is still naive as the focus was on functional aspects.

    A pull request for desktop support for the Csharp APIs has been raised and is currently under review. We do hope to get it merged once it is tested. @rachit@joy is beginning the integration with the Java JNI bindings provided by the core team. We are hoping to get the Java API covered as soon as possible for Android and desktop platforms.

    SAFE Client Libs

    We’re nearing the completion of the binding generator for Java and C#. We’re starting to work with the front-end and mobile teams to collect their input and their requirements for the bindings, and so far we’ve gathered some useful feedback on the C# bindings. With that feedback, we intend to cross off most of the TODOs during this week and start to integrate it with the actual .NET and Android/Java projects.

    @marcincompleted the automated tests2 for the Alpha 2 network, which we mentioned in one of the past updates:

    Besides manual testing, we intend to add some automated integration tests that will try to connect to the real network and use an existing account to do basic tasks

    These tests already helped us to uncover some obscure bug which we previously didn’t encounter, it concerns the apps revocation on the Alpha 2 network using the current master branch build of SAFE Client Libs. We’re actively working on pinpointing the exact cause of this bug and fixing it.

    Routing & Crust

    As data chains discussions and designs are nearing finalising phase, we decided to start coding features to better express the fundamentals and continue testing the design. Work has started at last. We have branched off Routing master into dev4. There one can now clearly see the fundamental types we will be using in the implementation of data chains. We’re currently working on a document defining and detailing the different parts of the changes that data chains integration brings and hope to have this ready and published on the Dev Forum next week (barring any nasty surprises early next week). In the interim, here are some of the key definitions that have been fleshed out in that proposal:

    Complete group: GROUP_SIZE peers with age >4.

    Elders: the GROUP_SIZE oldest peers in the section. If there is a tie, we use the tie-breaker rule for node aging.

    Tie-breaker rule for Node Aging: If there are multiple peers of the same age then XOR their public keys together and find the one XOR closest to it.

    Adult: a peer with age >4 in a section with a complete group.

    Infant: non-elder with age ≤ 4.

    Vote: a node’s detection of a network event plus its signature of it.

    Proof: a node’s id and its signature against a network event.

    Block: network event with a vector of proofs collected.

    Group consensus: a block containing a quorum of proofs of the elders.

    Section Membership: All the members of a single section, comprising the elders, adults and infants.

    Neighbours: sections we are supposed to be connected to differing in exactly one bit from us.

    Sibling: A section differing from us in the last bit.

    Corresponding code is also being integrated into the dev branch, for instance here are the events11Vote5 which is exchanged between Peers directly. A <code>Proofs and when it reaches a quorum of Valid voters the Block becomes valid. These valid voters are deduced from the chain order.

    As we had stated earlier, in a section we will usually have <code>GROUP_SIZE elders, some adults and some infants. Basically, elders are the oldest nodes in the section and it’s their votes that matter. If an elder is lost, an adult is promoted to take its place. Infants are the ones whose age is <=4 so they must stay long enough to see enough churn and get relocated for them to become an adult. Until then they do not contribute towards churn. This is because such young nodes have a much higher probability of coming and going offline thus the network does not rely on these nodes for general operation.

    With that brief categorisation in mind (much more detail will be available in the document we are currently formulating), the (local) events witnessed by a section will be voted by the elders and when a block is formed as described above, we will decide whether that should make it into the data chain or a state-container. For e.g. if we were accepting some peer who is going to be in the <code>elder group we put that block in the chain. If it was an adult or an infant we wouldn’t. This will reduce the size of the chain considerably because it’s elders who actually matter in the network in our current design. They are connected to the neighbouring sections’ elders etc. Infants can just come and go (because they have a small age, so they don’t have great incentive to remain online or they could just be spawned by casual/malicious users for appearing temporarily on the Network) and if we recorded all that, we would have ended up with an enormous chain. So one needs to “earn” one’s position in the chain - by becoming an elder and by that very virtue meaning that one has spent enough time in the network being a good node and also disincentivising them to act badly which would compromise their age which they’ve earned over a long time, few will wish to harm their nodes farming rate in that manner

    This is also a very important defense against any botnet type attacks, or large adversaries with boundless cash who may wish to attack the network in huge numbers. The network simply won’t allow such mass joining in short periods, but will instead ensure joining is a process the network controls through need. Therefore the network will require nodes when it needs them at the rate it needs them and upon accepting nodes, the network will monitor and validate the capability of the nodes (infants) over time as they mature and “grow up” to become valuable members of the network. A very natural design as inspired by the natural world, as always, if you cannot find your design in nature, it’s probably wrong. We are delighted we can and these steps are very powerful in their simplicity.

    Now that we have started with the code, we also thought it would make sense to quickly run some simulations to see things like what is the section size before we split (because we’re currently hoping to only split once we know both post-split siblings will have enough <code>elders and adults) so that if we end up with sections with a great many infants then we quickly devise ways to not have so. E.g. for a start we could say in a section with proper categorisation of peers into elders, adults and infants, don’t accept any more infant with age of 1 if we already have such an infant. @bartsmall" class="redactor-linkify-object">https://safenetforum.org/u/bar... simulation7 to test this and it requires some finishing touches but it has already given us some very useful results.

    @fraser@qi_ma are working on finalising the detailed data-chains document which will have the definitions and explanations and also rule-sets for various scenarios such as merges and splits. This will include the message flow, which makes looking at the algorithm much easier so that it can be reasoned about or improved and finally put to code and tested.

    In Crust, work has nearly finished with the integration of the <code>p2p library. There’s still a couple of bugs that we’re aware of in p2p (e.g. it doesn’t close router ports once it’s finished with them and so ends up hogging all the ports), but we expect these to be simple to fix. Once those PRs are merged and tested we will be able to start integrating our uTP library as well and Crust will no longer be limited to the slow and peer-to-peer-unfriendly world of TCP. A significant step to a secured multi-protocol, randomised port encrypted network library that will benefit a great many projects.

    <br />



  • MaidSafe Dev Update - Custom browser, Authenticator & API, Client Libs & More

    As many of you may already know, earlier this week we were able to release a new version of the SAFE Browser
    (v0.8.0), which changes were mostly related to the safe_app/DOM APIs
    while also improving the developer experience. This was accompanied by
    the release of new versions of the Email and Web Hosting applications,
    which have been updated to be compatible with the new API and
    Authenticator. See last week’s update for a summary of the API changes.

    Custom browser

    Today, we are releasing an initial version of the custom browser, which we first mentioned in this dev update It’s a proof-of-concept and has a few limitations as detailed below, but it looks very promising!

    Data chains

    Another thing to highlight this week is that we just published a post about data chains on the Dev Forum. It is a slightly deeper discussion regarding data chains, but does try and stay high level enough to be able to convey the important messages. Along with these aspirations, it is hoped that this document is also as brief as possible, allowing discourse and hopefully agreement on the fundamental aspects of the network.

    Marketing team expansion

    In addition to all the development work, we have also been fleshing out our marketing strategy over the past couple of weeks and have started implementing some of our plans. While it is not our intention to get into specific details at this stage, as to do so would put us at a competitive disadvantage in some cases, it will be much more community-centric, and in fact a couple members of the forum have already been helping with some early ideas and have been very useful sounding boards. Mapping out the strategy confirmed our suspicions that we would need more internal marketing resource and we are delighted to announce that Dug Campbell will be joining @SarahPentland at our HQ from next week.

    Dug has been a fairly central part of a small but flourishing Bitcoin scene in Scotland for a few years now. He organised Scotland’s first Bitcoin conference in 2014 and even spoke about it later that year at TedX Glasgow. He has been in touch with MaidSafe for the past few years and with his experience within other technology companies (Sky Scanner and Mii Card) in a variety of roles and his easy-going relaxed nature, he’ll be a great fit helping to plan and execute MaidSafe’s marketing and outreach strategy. Welcome aboard Dug!

    SAFE Authenticator & API

    We’re happy to show off an initial release of the custom browser. You can download it here

    0

    For developers, the browser is rebuilt around a React/Redux application for the UI. This allows for much more robust testing. The release process has been automated across all platforms, and we’re working towards some real end-to-end testing with the CI also. The browser also features a more comprehensive logging setup, and the ability to work against the live network without packaging, which should help speed up dev/debugging considerably.

    For end users, the browser features the DOM API and a new (currently very simple) setup for authenticating apps. You should be able to authenticate both web apps and desktop apps using the latest version (0.5.3) of safe-node-app package.

    Known limitations:

    • No localhost support in this release
    • Placeholder authenticator messages (the prompt doesn’t show the app’s info and/or permissions requested as the current authenticator does)
    • Potentially slow startup (network connection occurs on startup as opposed to first fetch)
    • No reconnect notifications (if you start without the invite IP updated, you’ll need to restart, for example)
    • UI has not been a focus as yet
    • No page loading indicator right now

    The comments plugin example web app has been updated by @srini to be compatible with the latest changes in the DOM API as well, thus it can now be used with the latest SAFE Browser v0.8.0. The two versions previously published on the Alpha 2 network were updated and can be accessed at the same locations:

    The CI scripts to automate the packaging/release process of the browser with mock/non-mock routing were implemented by @hunterlester and we’ll be testing them so we can for our next release. We now plan to automate this process for the example applications.

    Some of the tasks we started this week are related to making sure all our applications work as expected also on 32-bit platforms. The safe_client_libs already supports them and we just need to work on some Node.js dependencies that seem to be causing problems in such platforms. In addition to this, we will be working on some minor internal refactoring in the safe_app_nodejs code, along with the addition of some new binding functions that are already exposed by the safe_client_libs, e.g. to get the app’s own container name.

    The pull request for desktop support for the Csharp APIs has been merged. The API scope is also being enhanced in parallel. We will have to update the mobile Authenticator and also the messaging app with the new Csharp API to test end-to-end before we publish the NuGet package. The JNI bindings are being tested and integrated. There are few points like mock support in the JNI bindings which are being looked into. We will soon be starting to integrate the bindings for the Java and Android APIs. @joy has been away this week for personal reasons and once he resumes work we expect progress there to continue.


    SAFE Client Libs

    This week we worked on, finished, and merged an integration test which logs into the real Alpha 2 network and performs data operations. Writing this test has already paid off as it allowed us to find and fix a revocation bug (the fix is in the same PR). What happened was that the revocation flow was interrupted after deleting an app’s entry from the access container but before removing the app from the revocation queue. This resulted in all subsequent revocation attempts trying to handle this app from the queue that couldn’t be found in the access container. Now, if an app is in the queue but not in the access container, we simply exit early without doing anything and allow the app to be removed from the queue.

    Of course, the real purpose of the new integration test is to guarantee data compatibility with the network. We ran the test and verified that the master branch is data-compatible with the Alpha 2 network, allowing us to safely deprecate the alpha-2 branch. All further code changes will go into the master branch with confidence that they will not break compatibility.

    Work on the C# bindings generator has wrapped up and is currently undergoing review. Once finished, this generator will help the developers of the SafeApp mobile bindings to reduce most of the low-level repetitive boilerplate, allowing them to focus on more interesting parts.

    Some of the manual patches we’ve needed in our lower-level crates (namely config_file_handler and get_if_addrs) when compiling them on mobile platforms are being addressed upstream and corresponding PRs have been raised. These are currently undergoing review. We are also planning to set up an automated build system for the mobile libraries, to streamline their releases.

    Java bindings are almost completed too. We have written some simple tests and confirmed that the basic functionality is working. Soon the front-end team is going to start integrating it into the mobile project targeting Android. Currently, we’re working on automating the bindings generation by including it into the SAFE Client Libs build pipeline (there’s a pull request for C# bindings and Java bindings1 are underway too).

    Routing & Crust

    Today we are putting out a forum post for our proposal of the Data Chains. We have managed to simplify the chain quite a bit. It now has only a few states for the Elders - Live, Dead and Gone as explained in the post. They follow a natural constraint that Dead or Gone must be followed by a Live. This means we will always see these events happening in pairs. This is a simple but wonderfully powerful tool for chain validation. Peers on the Network cannot delete any individual Block from the chain, if they do so the chain-validity will no longer hold and it’s easy to verify this. This aspect is particularly important because we say a peer can be given only one copy of a chain by anyone in the Network and it can implicitly trust that to truly contain the correct sequence of events without any of them missing in between, provided there is a point in the chain that the peer already trusts. For a peer that was never on the network before, this could be the genesis block. For one that was in the network before, it could be the block it had before it left the network. All these will become more apparent when you look at the forum post. We have tried to make it concise and precise.

    Triple Quorum

    This is one of the ideas we have for applying to Secure Message Transfer. When we get a message block which is valid by the virtue of it being signed by the Quorum of the previous section, we have to vote on this message ourselves and obtain a block and pass it along. However, if our section was undergoing a merge, then our Elder group would likely be undergoing frequent changes. This might not have been realised by the previous section while it sent out the message block to us. It might happen that some of the peers it thought were Elders are now Adults. Under these circumstances these Adults relay these blocks (or votes that they get from Elders of their own section which still think of it as an Elder) to what they consider should be the current set of Elders. The triple quorum rule means that an Elder getting such a Vote or a Block from an Adult, which it would normally ignore, will consider it valid if that Adult was an Elder in either the pre-merge own-section or the pre-merge sibling-section. This allows us to not drop the message when it’s transiting from a section undergoing a merge.

    Crust has been evolving at a steady pace. uTP is almost there and we should be able to test it next week. p2p integration is done which also resulted in codebase size reduced by 7% and more unit tests. Crust is developed hand in hand with p2p which also got some attention. There was a bug on Windows systems: if we tried to recv_from() a closed UDP endpoint (Windows would detect that if previously we sent something to that address), it would raise ConnectionError. So instead, we ignore the error and keep receiving datagrams. Also, UPnP port forwarding rules were infinite, but now they expire in 5 minutes. That should manage router resources more economically. Although, if we notice connections getting dropped, that might be changed in the future. In addition, now users have finer control over port forwarding over UPnP: one already could disable UPnP use globally. But now it’s possible to enable/disable it for rendezvous connections only and that allows us to avoid needlessly overloading routers. Some routers have a maximum limit of ports forwarded by UPnP. Hence, being able to better utilise them can definitely save us some trouble in the future.




  • Maidsafe Dev Update 

    Peruse (the custom browser that will eventually replace SAFE Browser) has been updated, you can download the latest version here. It adds localhost support, as well as the ability to open http links in your clearnet browser of choice automatically.

    0

    Keep in mind that this is a proof of concept and has a few known limitations:

    • Placeholder authenticator messages (the prompt doesn’t show the app’s info and/or permissions requested as the current authenticator does)
    • Potentially slow startup (network connection occurs on startup as opposed to first fetch)
    • No reconnect notifications (if you start without the invite IP updated, you’ll need to restart, for example)
    • UI has not been a focus yet
    • No page loading indicator right now

    Community Engagement Program (CEP)

    Followers of the project and regular visitors to this forum will have seen a lot of discussion on the forum about marketing, and as mentioned in previous updates this is something that is getting much more focus and resources with @SarahPentland joining the team a couple of months ago and @ketch (Dug Campbell) joining this week. There are a couple of areas that we will be focussing on, but one is much more community involvement. The SAFE Network must have the strongest community in the space and even after 3+ years we continue to be astonished by the energy, intelligence, commitment and creativity of the people on this forum and it would be almost criminal for us not to use these skills to all our advantage. That is why next week we are bringing back the Community Engagement Program (CEP) as the first step in this process.

    The purpose of the CEP is to match MaidSafe requirements with skills in the community, taking the form of a community funded ‘kickstarter’ like process to encourage community members, individuals and teams to bid for projects. Before this was more focused on technical requirements and longer standing forum members will recall that the CEP gave birth to the SAFE Browser, currently part of Alpha 2, and also led us to hire @joshuef. Our intention with this relaunch is to not only focus on development tasks but to also incorporate non-technical requirements, such as design, animations, videos, content management…etc…Sarah and Dug are currently working on an explainer video and some other resources to make the process clearer and to educate newer members about its existence and we look forward to launching this next week.

    SAFE Authenticator & API

    The DOM API playground tool4 was updated to the latest changes in the DOM API and it’s now fully compatible with the SAFE Browser v0.8.0. This update included a refactor where the code snippets were changed to make use of async/await as we hope it should make it simpler/easier for developers trying to understand them and to then use them in their own apps. Although intended for mock development, it has also been published at safe://codeplay.discover3 for initial discovery. Please keep in mind that it can quickly consume mutation requests.

    The scripts to automate the SAFE Browser packaging process to generate versions for mock and non-mock routing have been merged and we are currently testing them to have them ready for our next release of the browser.

    During the last few days, we were also going through the list of bugs reported for the browser and plugins, not only those reported on GitHub but also some others discovered internally, and we started fixing some of them to be able to release a patch for the browser soon.

    Peruse has been updated, you can download the latest version 0.2.0 here26. It adds localhost support, as well as the ability to open http links in your clearnet browser of choice automatically.

    We’ve dropped the custom localhost protocol (as found in the SAFE Browser), as this was causing a number of issues in Electron. Now http://localhost:<port> and http://127.0.0.1: are whitelisted HTTP URLs for use in Peruse. This sidesteps these Electron issues and enables more robust local development (most notably hot-reloading of your applications).

    Otherwise, this week has seen a lot of refactoring of the Peruse store to enable syncing across browser windows and increasing test coverage, which puts us in a good position to implement history and favourites to round out a bit more of the basic browser functionality.

    As a first step in integrating the JNI bindings, we are implementing a few minimal APIs, setting up the test suite and testing the desktop JAR files from a quick example app. Once these steps are covered, we will be starting to expand the APIs.

    The Java project is structured such that the api1 project will be a dependency for safe_app1 (desktop) and safe_app_android3. Only the platform-specific variations will get coded into the desktop or Android project. For example, the loading of the native library is different in desktop and Android. Hence this part will be handled in the platform-specific project while the majority of the API code will reside in the api project.

    The progress with Java bindings has been good. We are well placed to cover these three steps and get it merged with the upstream master by this week.

    A pull request1 adding ReSharper code inspection to the AppVeyor CI script and removing identified linter issues has been merged.

    SAFE Client Libs

    As we keep supporting more and more platforms and processor architectures, maintaining their releases has become tedious. To streamline the process, we came up with a plan to automate it as much as possible. The first step of this plan was to write a script3 which automatically builds and packages all necessary libraries and other support files for each supported platform and architecture. This script is now ready and is undergoing review and testing. After that is done, the next step will be to include it in our continuous integration system.

    Today we are releasing a first community preview3 of the auto-generated Java bindings. It consists of two parts: the Java files, defining the API, and libsafe_app, which glues JNI functions to the Rust implementation of SAFE App. For now, the released version is available for the x86_64 Linux platform only, but as soon as we’re done with the build automation scripts, it should be available for all the platforms that we support.

    We’ve also received some valuable feedback from the front-end team and started implementing it already, so there will be more news and updates in the coming days. The feedback concerns the minor stuff, so the Java bindings should be wrapped up pretty soon.

    Routing & Crust

    As we’ve been drilling down a bit further into the details of the proposed implementation of Data Chains1, we’ve come across some problematic examples which have resulted in further team discussions and refining/updating the protocols. As expected, the chief culprit of the problems is the section merge procedure. That’s what happens when a section of peers becomes so small that it needs to join up with one or more neighbouring sections.

    This has proved troublesome in the past, and we’ve learned a lot from previous work about how this can go wrong. That experience is invaluable, it’s made us all very aware of how easy it is to overlook edge cases which can hurt the network. This is just a symptom of decentralisation: if all the peers agreed simultaneously about how the network was changing around them, it would be relatively trivial to handle. However, at a given moment, since half the peers in a section can “see” one thing and the other half see something slightly different, it makes achieving eventual consistency in this case particularly tricky.

    As a result of this, we’ve not progressed the code as we’d have hoped - instead, we’ve been largely focussed on resolving the remaining issues in the merge protocol.

    We are also delighted to have our work closely scrutinised by forum members. Special shout out to @mav for his interest and contribution17. Seeing so much interest from the community is awesome and gives us even more motivation to work on the project 

    @mav’s interest in the project also led to him discovering a serious bug in our simulation - it was caused by relocated nodes not being actually removed from the source section, which resulted in their duplication and artificial increase in the number of older nodes. The results of the simulation changed dramatically after this bug has been fixed. This exposed a flaw in our current design which made the network accept barely any new peers. Fixing this flaw will be an important task for the near future. Summarising, @mav’s efforts are quite a significant contribution to the project - much appreciated! 

    uTP support has finally landed in Crust’s dev branch. uTP is a UDP-based transport protocol which has several advantages over TCP. Its congestion control algorithm is designed to use all the spare bandwidth on an internet connection while yielding to anything else which tries to use the same connection. This makes it a perfect protocol for peer-to-peer downloads from the SAFE Network. UDP hole-punching is also much more reliable than TCP hole-punching, meaning that we will be able to make direct peer-to-peer connections more often than we were able to with just TCP.

    Some parts of uTP support still haven’t made it into Crust. Specifically, the rendezvous server implementation used for UDP hole-punching still needs to be integrated from our p2p crate. But we expect this to be completed within days.

    The next stage for uTP support in Crust is, of course, testing. We plan to do some more internal testing first to make sure there are no glaring bugs, then release binaries to the community so that we can get some testing done in the real world.



Looks like your connection to Cryptocentral was lost, please wait while we try to reconnect.