Implement Web Storage API on Native

I need BJS Native to be able to store state when the app is closed. Just being able to store simple key value pairs, would help greatly.

Using a pre-existing browser feature not only relieves the need for wheel re-invention, it means the code could run both places. Never a bad thing. Think the “session” data could just be held in RAM.

I could go on with use cases, if asked.

cc @bghgary

Hmmm if you mentioned React Native I think what you’re proposing wouldn’t be a ridiculously heavy lift, but for Native it’s different.

First thought is that we’d need to implement platform-specific interfaces for storage, but if the storage and indexedDb implementations are available to bring in and hook up, that reduces the work and complexity.

Alternatively, is it possible to provide your own storage mechanism in your Native app? The (de)serializers don’t care how their JSON gets supplied, so would perhaps writing json text files be a viable approach?

1 Like

Is there any reason you can’t use existing browser storage solutions such as localforage?

1 Like

I looked at the localforage on the link. It is for wrappering Web Storage to emulate indexDB for browsers which did not have it. It does not directly write anything itself, which is needed.

In the emulation spirit though, just the smallest possible entry points in JS to: writeFile & readFile would work. The dev could wrapper those 2 calls into emulating Web Storage API with the help of JSON.

I want to avoid writing native directly & not just because I have not been a full time C dev since 1995. I have something in mind for Native written in JS, which multiple people can use. Putting in a home brew Native dependency seems a little clumzy.

Sorry, I wasn’t really sure what’s available in the native environment. I assumed it’s using something like electron which would have access to browser APIs.

So you’re able to write files to the system? What do those function signatures look like?

One possibility is to implement PUT for XMLHTTPRequest. We already have an implementation for GET. This will allow writing to disk using app:/// or file:/// protocols.

2 Likes

That works for me.

Anyone want to implement it? It will involve adding corresponding PUT code for each platform to the UrlLib dependency and hooking it up to XMLHttpRequest polyfill.

I can follow patterns. I added missing webaudio nodes to Exokit, when I was still considering it. There I was going from like 8 to 16. A little more guess work when going from 1 to 2.

While I might be able to Windows, then try Android if that worked. At first look, it seems like the platforms facility is just being called like, winrt/Windows.Web.Http.h.

Feel Apple nor Unix are not doable by me.

Well after a morning yesterday of getting a fresh Repo, changed to VisualStudio 22 from 17, & updating to a supported version of CMake, I managed to do a VS build & get a running un-modified playground in the afternoon.

I also finished my first cut of the code changes this morning in the areas you indicated. I get no visual errors in the VS tab for the file, but get build complaints from arcana:

1>UrlRequest.cpp
1>C:\BabylonNative\Dependencies\arcana.cpp\Source\Shared\arcana\threading\internal\callable_traits.h(40,1): error C2064: term does not evaluate to a function taking 0 arguments
1>C:\BabylonNative\Dependencies\arcana.cpp\Source\Shared\arcana\threading\internal\callable_traits.h(40,1): message : class does not define an 'operator()' or a user defined conversion operator to a pointer-to-function or reference-to-function that takes appropriate number of arguments
1>C:\BabylonNative\Dependencies\arcana.cpp\Source\Shared\arcana\threading\internal\callable_traits.h(93): message : see reference to class template instantiation 'arcana::internal::invoke_result<UrlLib::UrlRequest::Impl::SaveFileAsync::<lambda_6145541bc23ba1dd93ae32ac95755190>,std::integral_constant<bool,false>,void>' being compiled
1>C:\BabylonNative\Dependencies\arcana.cpp\Source\Shared\arcana/threading/task.h(97): message : see reference to class template instantiation 'arcana::internal::callable_traits<CallableT,void>' being compiled
1>        with
1>        [
1>            CallableT=UrlLib::UrlRequest::Impl::SaveFileAsync::<lambda_6145541bc23ba1dd93ae32ac95755190>
1>        ]
1>C:\BabylonNative\Dependencies\UrlLib\Source\Windows\UrlRequest.cpp(244): message : see reference to function template instantiation 'auto arcana::task<void,std::exception_ptr>::then<const arcana::`anonymous-namespace'::<lambda_86fcb5e973fcdacccc4842e1526a0ecf>,UrlLib::UrlRequest::Impl::SaveFileAsync::<lambda_6145541bc23ba1dd93ae32ac95755190>>(SchedulerT &,arcana::cancellation &,CallableT &&)' being compiled
1>        with
1>        [
1>            SchedulerT=const arcana::`anonymous-namespace'::<lambda_86fcb5e973fcdacccc4842e1526a0ecf>,
1>            CallableT=UrlLib::UrlRequest::Impl::SaveFileAsync::<lambda_6145541bc23ba1dd93ae32ac95755190>
1>        ]

I have never used arcana before, so not sure where to go from here, so I am just going to list my changes from lowest call to highest for comment.

Made a SaveFileAsync method very similar to the existing LoadFileAsync, but smaller. I suspect I am having an issue that WriteTextAsync() does not actually return text, so the assignment of m_responseString should not be done. I do not know how to remove this:

arcana::task<void, std::exception_ptr> SaveFileAsync(Storage::StorageFile file)
{
    return arcana::create_task<std::exception_ptr>(Storage::FileIO::WriteTextAsync(file, m_putContentsString))
    .then(arcana::inline_scheduler, m_cancellationSource, [this](winrt::hstring text) {
        m_responseString = winrt::to_string(text);
        m_statusCode = UrlStatusCode::Ok;
    });
}

This is called in my revised SendAsync method with the added putContents arg. The code is very similar to the Get, but without a section for doing it over the network.

arcana::task<void, std::exception_ptr> SendAsync(std::string putContents)
{
    try
    {
        if (m_method == UrlMethod::Get) 
        {
            ... // same as before
        }
        else // Put part
        {
            m_putContentsString = winrt::to_hstring(putContents);

            if (m_uri.SchemeName() == L"app")
            {
                return arcana::create_task<std::exception_ptr>(Storage::StorageFolder::GetFolderFromPathAsync(GetInstalledLocation()))
                    .then(arcana::inline_scheduler, m_cancellationSource, [this, m_uri{m_uri}](Storage::StorageFolder folder) {
                        return arcana::create_task<std::exception_ptr>(folder.GetFileAsync(GetLocalPath(m_uri)));
                    })
                    .then(arcana::inline_scheduler, m_cancellationSource, [this](Storage::StorageFile file) {
                        return SaveFileAsync(file);
                    });
            }
            else if (m_uri.SchemeName() == L"file")
            {
                return arcana::create_task<std::exception_ptr>(Storage::StorageFile::GetFileFromPathAsync(GetLocalPath(m_uri)))
                    .then(arcana::inline_scheduler, m_cancellationSource, [this](Storage::StorageFile file) {
                        return SaveFileAsync(file);
                    });
            }
            else
            {
                throw std::runtime_error{"Network PUT request not currently supported"};
            }
        }
    }
    catch (winrt::hresult_error)
    {
        // Catch WinRT exceptions, but retain the default status code of 0 to indicate a client side error.
        return arcana::task_from_result<std::exception_ptr>();
    }
}

There are also 2 little adds:

  • add a check for a Put in ConvertHttpMethod()
  • list the member winrt::hstring m_putContentsString{};

I’ll leave the changes to XMLHttpRequest.cpp for later. Am I even close?

Can you put this in a branch on GitHub so that I can see a diff? It’s a bit hard to read with the forum. :slight_smile:

arcana::task<void, std::exception_ptr> SaveFileAsync(Storage::StorageFile file)
{
    return arcana::create_task<std::exception_ptr>(Storage::FileIO::WriteTextAsync(file, m_putContentsString))
        .then(arcana::inline_scheduler, m_cancellationSource, [this]() {
            m_statusCode = UrlStatusCode::Ok;
        });
}

Probably something like this.

1 Like

Yeah, I guess I have spellchecker on somehow. There are so many squiggles & text wrapping I could almost not make the post (did just help me spell squiggles though).

It looks simple after you see the answer :slightly_smiling_face:

It is beyond that now. Am now down to stuff in image, & ScriptLoader not passing a putContents arg. Putting empty string.

Ok, I have a built / running exe again. Not running through the new code yet. Need a little guidance about testing. Now that XMLHTTPRequest is bi-directional, maybe there can be a self re-enforcing test pair:

1- wiite a file with a single letter in it
2- do a read of that file in the “load” event listener of the step 1 write
3- check the value of the read is correct in the “load” event listener of the step 2 read

Where are tests / how are they run?

There are a few unit tests for XHR. All the unit tests are here: BabylonNative/Apps/UnitTests at master · BabylonJS/BabylonNative (github.com)

You can add the new tests here: BabylonNative/tests.js at master · BabylonJS/BabylonNative (github.com)

You run the tests just like the Playground app except run the UnitTests app.

Yeah, I just hit the little Green play icon in VS :roll_eyes:. Now that you pointed out that there are even multiple targets, I see them. Is the object of getting the play button to switch to “UnitTests”, getting it to be the one in Bold?

plaground app

Yes, you can do this by right-clicking:

Or you can just run it directly there:

1 Like

I can make these comments over in the repos too if it helps.

In the future, we might want to handle other verbs than GET and PUT which aren’t currently implemented, so would it make sense to account for that possibility later by refactoring your if conditional statements to e.g., switch?

1 Like

Thanks, I changed to using a switch. The good news is all prior tests pass. My test only works if you make sure the file already exists, otherwise the UnitTests app hangs.

Here is my test (I also added a contents argument to createRequest():

    it("should write a local file", async function () {
        const contents = "This file is an artifact of XMLHTTPRequest PUT testing";
        const putXhr = await createRequest("PUT", "app:///delete.me", contents);
        expect(putXhr).to.have.property('readyState', 4);
        expect(putXhr).to.have.property('status', 200);
        const getXhr = await createRequest("GET", "app:///delete.me");
        expect(getXhr).to.have.property('readyState', 4);
        expect(getXhr).to.have.property('status', 200);
        expect(getXhr).to.have.property('responseText').equal(contents);
    })

Think this is a result of copying the code for doing a GET, where you obviously need a file to exist. This is too much of a pattern shift for me.