YaCy on a Kiosk

Tom_Booth · 22 December 2020 02:00

I’ve recently been working on a Kiosk system for tablets to place in coffee shops and other similar semi-public areas. Mostly to help promote a “reader ads” type (or pennysaver type) site for local business, news, classified ads, etc . I’ve been developing.

The idea being, people in these public gathering places, coffee shops etc. can brows the internet on these tablets, the way people might casually pick up and brows a newspaper or magazine, while eating brunch or sipping tea or whatever.

The tablet hardware would be “locked down” so as to somewhat restrict what people can do, what websites they can visit and so forth, as with most any Kiosk software.

Anyway, with that kind of set up or “clustering” of similar hardware in various locations: laundromats, coffee shops, luncheonettes, etc. YaCy could be running on all these tablets as the default search engine.

It is not hard to find piles of cheap tablets around these days.

Kind of like publishing a local newspaper with classified ads to generate revenue, but without the overhead of printing and distribution.

Anyway, I think it would be very cool, and could run on some tiny linux kernel with just a web browser, so as far as the device, there wouldn’t be much there to hack.

zooom · 22 December 2020 05:33

Cool Idea.

Which kiosk software do you use? Stuff like: https://github.com/coderbunker/kiosk-android ?

I think you may need an intermediate logic to provide the local news and the advertising content.

I am working on such kind of a news portal based on a mix of regional and international news. YaCy is heavily involved to crawl the newspapers. Another one is precalculating the proper content.

The “App” will be a mobile-first website so all you need is a Kiosk on a Tab which only allows a browser.

Do you already have a concept to finance the tabs and a marketing plan?

Cheers
Markus

Tom_Booth · 23 December 2020 01:18

I’m still researching kiosk software options. There seems to be dozens and I don"t really know where to begin. The only one I’ve managed to get working at all is just a Firefox browser plugin. It works fine, but was never finished and hasn’t been updated in years. It can be exited by just hiting control/alt/delete so is not much use, as is.

I don’t think I’ve looked at kisk-android, but I will now.

The local news is more or less just a static html website, not associated with the search engine. The classifieds are also mostly static but with a form for submitting new free reader ads. Other bigger more complex local ads would likely be paid as it would require some work on my part.

It would also generate some work building websites to link to the ads.

All the news and advertising, in other words, would be entirely separate from YaCy. running on a seperate server somewhere.

I’m not sure what you mean by “a Kiosk on a tab”.

My plan is to buy several tablets. Have a minimum install of some bare-bones Linux with a web browser.

The tablets’ web browser would be “locked down” so as to default to, or open with, my local news/classified site. And a limited number of other local sites could be whitelisted. Local businesses, stores, services, the village office, library etc.

My partner and I have been working on this for a long time, and have purchased a commercial space downtown and will soon have an office open to the public - so all this is separate from and essentially has nothing specifically to do with YaCy.

But, given all these tablet “kiosks” to be placed all over the area. They, theoretically, could be running YaCy in the background and since YaCy runs in a web browser, localhost/YaCy would be, or would appear to be just another site available / whitelisted in the kiosk/browser.

It would mostly be a local community thing. So the YaCy instance(s), would not have to index much more than the various websites belonging to members of the local community.

There would be something like a link or button on the kiosk’s initial “splash page” providing access to the local news, classifieds, business directory, village office, etc. and in theory, local search/YaCy.

I’m still not sure what you mean by “tabs” when asking “Do you have a concept to finance the tabs…”

It would be more a community service than anything.

The YaCy local search would, I think, probably be implemented as an intranet of sorts, only accessable from the physical kiosks, not from anyone’s smart phone, though, in researching various kiosk options, I do see there are some paid services where a WiFi server can be set up in a business so that people can access the “virtual kiosk” on their phone, which is an interesting concept, but not really what I had in mind.

Income would be primarily from website design or “webmaster” services and advertising on the classified page, though I’m not anticipating it generating much if any income. As I said it would be a community service, but I would need to make enough to cover expenses.

By kiosk I mean physical devices set up in public spaces, like coffee shops or laundromats and convenience stores etc. Not a tab on a browser on a personal mobile device.

Tom_Booth · 23 December 2020 06:20

Kiosk-Android seems to be specifically for displaying a promotional video on a kiosk.

Something like “JustBrowsing” which I’m downloading now might work. the 64bit firefox version ISO is just a little over 266MB and Based on Arch Linux.

There is a long list of other possible options.

The only option I’ve found so far that appears to still be actively maintained though is Porteus, which looks quite interesting in that there is also a server for remote administration of the Kiosks.

Standard Kiosk software available however seems mostly to be of the “Live” type. That is, running YaCy with any sort of persistent storage could be an issue, but, interestingly, porteus offers a “custom build” service. So if the price is reasonable, I might just ask them to make me a kiosk that can run Yacy, and save myself all the headaches (or fun as the case may be but mostly I may not have the time). They do mention 0n the website it is possible to run an Apache web server and various other similar possibilities, which anyone, in theory, could do themselves, or they do offer the custom build service, but I imagine that could get costly, don’t really know, but the automatic update service is 40 euro, or about 50 US dollars per kiosk, I think(?)

I’m looking at so many different options and possibilities, I really don’t know which way this is going to go. Porteus seems almost overkill for what I need, and I don’t really like the idea of being tethered to any kind of third party support but it is at this point a very attractive option. It is essentially free, without any of the extra support options.

Ubuntu 18.04 apparently has kiosk.sh though I’m not entirely sure I could manage the setup but maybe. Here’s a video on how to do that, anyway:

I’m not really a Ubuntu user though. and I’m not sure this is suitable. But I’m not sure why it wouldn’t be. Ubuntu is probably a little (or a lot, about 2.5 Gig compared with 100 Mb for Porteus) bloated for running a simple kiosk with just a web browser on a cheap tablet…

The basic idea though is to have all these public kiosks with yacy as the default search engine which could form a “cluster” so the load on any one machine would be relatively light.

Having it all on a local wireless mesh network would be cool.

I’m probably getting in over my head. But my long term ideal would be a free internet where such local “clusters” could be interconnected to form larger and larger networks (without paid access providers, or giant “big tech” portal search monopolies, or dependence on the telephone networks. or at least fewer or less such dependence)

Anyway, I’ve downloaded a variety of potential ISO’s. so I’m ready to get off here and start trying them out.

Tom_Booth · 23 December 2020 09:18

This looks like another possibility. Appears to be very robust with recent development activity.

https://libki.org/

But is perhaps more on the order of a time management system for general computer use, giving full access to the os and programs. Regardless, if the “client” is only running a web browser and yacy on a core linux it might well work I suppose.

Tom_Booth · 23 December 2020 23:55

Well I tried many different downloads for various kiosk programs or ISO’s and

Many were old and outdated, unsupported, would not load or boot at all (not on my Acer laptop anyway) or would try to boot, but fail and fall back into command line mode. Which, someone who knew what they were doing might get somewhere with.

Anyone have any idea what to do when confronted with a prompt that says:

[sudo] password for toeknee

I could find nothing online that provided any clue.

The only exception so far was Justbrowsing: http://justbrowsinglinux.com/

Which at least works, boots up and loads from a USB drive and functions, more or less as intended, I guess. but not really very customizable and quite outdated in that the default bookmarks and such pointed to many no-longer-existant resources.

Then I booted up Porteus.

Porteus is kick ass awesome.

I’m actually posting this through the built in “test” browser (in this case Firefox was my selection) downloaded (into RAM) and made available during initial configuration.

There are so many options and features, I don’t think there is anything that Porteus can’t be configured to do in terms of operating and managing any kind of Kiosk whatsoever.

I can’t really get it all up and running as I haven’t booted up the server, but wow, it is hard to believe this is a free program. Just amazing.

I may be a little premature in my enthusiasm, but, Porteus scanned for a wireless connection and no problem getting online right away. The configuration options are brilliant. It just seems perfect for everything and anything I wanted to do, and more that I never thought of.

The only problem I see is the only firewall options are either ON with the default settings or OFF, so no obvious way to only open port 8090, but my home router is basically a hardware firewall, so I guess setting that to OFF is OK, but elsewhere, in a coffee shop or somewhere with their own router, the router or Yacy would likely have to be configured to open a port or use a different port or something.

After the Porteus client is configured, the modified ISO can be burned from the Live system onto a USB drive, however, I did not have a blank USB to burn it to, so I will have to run the configuration again, but I’m very anxious to get this up and running now.

I actually booted Porteus from the same Easy2Boot drive I have all the other ISO’s on that I tried, so I’m working/posting/internet browsing from a live Porteus system running in RAM with a Firefox browser, just downloaded into RAM a moment ago, which I think is rather awesome in itself. Now if I can figure out how to install YaCy as well, but I suspect that is not going to be quite as easy as pushing a button to download Firefox was (which is a built in feature, of course).

To be fair, I have not tried all of the other available options, like Libki. There are also many Windows kiosk programs I haven’t tried as yet, but I’m not really interested in building on a proprietary OS for this project.

All in all though, I don’t think I will find anything to match or surpass Porteus. It seems to be as perfect as anything could be, I’m really blown away.

Tom_Booth · 25 December 2020 22:35

In my personal email, to which messages from the forum are sent, there is an earlier post mentioning you might be able to provide a news “Tab” for the Kiosk.

I am interested, though I’m not entirely sure what you have in mind. I find no means of PM here.

Anyway, I had an awful time trying to figure out how to integrate YaCy with Porteus.

Porteus is apparently based on Slackware, which had its own software installation methodology, apparently different from other Linux flavors, and seems to require a different version of Open Java. I haven’t the time lately to try to figure it all out, so I ended up contacting Porteus, out of curiosity, just to see what the cost of a custom build would be.

By the time I received a reply shortly thereafter, the customized Porteus + YaCy ISO was done and ready for download for a nominal fee.

I have yet to boot it up, but there are apparently no restrictions on distribution, so if anyone wants a copy of the ISO to try, I can upload it to a server.

For reference, from correspondence with Porteus:

MD5 sum: a09715c998acdfc2bed7ec89699ac319

Yacy is installed in /opt/yacy folder so if you want to make it persistent then please use following parameter:

persistence=/opt/yacy

I believe, or as far as I’m aware, the folders are only accessible via the Porteus Kiosk server, or possibly by unpacking the ISO (?)

I have yet to get this set up and running. So far I am very pleased with the Porteus customization service. I was not expecting it to be so incredibly fast and affordable.

Is that per unit installation?
No - its for unlimited installations.

By sending the screenshot, is that to say the work is already completed?
Correct.

Tom_Booth · 26 December 2020 03:48

I think I must be about as pleased as a person could possibly be at this point.

The Porteus+YaCy ISO boots up in configuration mode. And so is still fully customizable. When finished it needs to be burned to a blank boot disk. I didn’t have a blank USB so I used an old Ubuntu boot up USB I don’t use anymore that is on a $5 discount flash drive I picked up a dozen or so of several years ago.

Now fully up and running in senior mode.

There seems to be a few additional configuration options included with this ISO that allowed setting Porteus to load YaCy at boot time. Just enter ./startYACY.sh in the field available during configuration. Burn, boot and good to go. The kiosk boots up into a browser and starts YaCy.

Also, I set YaCy as the start screen.

Yacy indicated right away though, that there was not enough disk space, but I neglected to configure persistence.

I’ll probably go through the process a dozen times before I end up with everything set up just right. There are so many configuration options, I just left most of them at default setting so I could just get booted up to confirm that the basics all load and function.

Next step, I guess, is to acquire some suitable hardware. For the public kiosks as well as for the administration server.

Merry Christmas all.

zooom · 26 December 2020 14:18

Why should anyone these days pick up a tablet as everybody is staring only at their mobile phones the whole day?

The idea reminds me to years ago sitting in hotels abroad being offline due to no wifi available.

How do you generate money? If ad-based, you need something starting from 10 tendhousands klicks per day.

How do you make sure thet the tablets will not be stolen?

Finally: Cheap tablets means slow hardware. Just the opposite where you wanna run a YaCy instance.

zooom · 26 December 2020 14:43

Sorry if “tab” was misleading. I meant “tablet”. What I am doing is, 1st crawling as many newspaper sites as I can get, which is some thousand here in Europe. 2nd: Use “searx” as the metaengine to grab the latest news from everything besides G*** and 3rd: Do some linguisitic statistical computations to extract the unique words which classify the headlines. If you now want to generate local news, you must find some local tokens (search terms) which you feed to your searx instance, which has your YaCy instances built in with the latest news you crawled by your own. The “app” finally is only a card-like mobile-first website which displays the latest data.

Tom_Booth · 27 December 2020 04:39

Card-like ?

Would that be displayed within an iframe or some such?

examples: https://allwebco-templates.com/support/S_script_IFrame-NewsScroll.htm

Tom_Booth · 27 December 2020 04:52

Anyway, this is interesting.

I thought it would be rather difficult to set up if I wanted to allow people to access the YaCy kiosk from their phones.

Apparently, the only real problem there might be limiting access.

I’m able to access the kiosk from my Android phone’s browser and conduct a search. Seems very fast.

Maybe just get a domain or subdomain or something and set up dynamic dns?

Or does YaCy have that built in too? I would not be surprised.

Apparently my kiosk can be accessed by anyone anywhere in the world with an internet connection.

Tom_Booth · 27 December 2020 05:03

Admin area easily accessable as well:

Strike that.

There is only very limited access to the admin area. Cannot shutdown or make other system changes. Nice.

It did seem I was able to gain full admin access from another computer on our home router though.

Tom_Booth · 27 December 2020 05:08

This kind of answers my dream of how to open up Yacy to smart phones!

YaCy looks great and works amazingly well on an android phone, accessed via a Porteus Kiosk.

Insanely easy. Just boot the ISO from a flash drive.(on any device that can boot Porteus Kiosk, I’m using an Acer laptop currently) then access the URL reported in the YaCy admin area.

I’m still waiting to hear from anyone regarding the question of can the Kiosk be accessed from anywhere.

It is theoretically possible this is only making YaCy available locally on my home network.

Kind of interesting I guess

I searched simultaneously for “leaves” on both the laptop running the kiosk and my phone accessing the kiosk and both returned the exact same results in the same order.

Image search on phone via kiosk works also and seems very fast!

I did have to reduce the size of this screenshot to upload to the forum.

Tom_Booth · 27 December 2020 06:40

Quick launch icon:

Screenshot_20201227-013459_crop_3_resize_14

I need to stop.

Don’t ask me why the #4(?)
No idea.

zooom · 27 December 2020 13:00

Try this:

Tom_Booth · 30 December 2020 02:32

Ok, I’ll take a look.

FINALLY, It looks like I’ve managed, with some help from Porteus Solutions, to configure the YaCy kiosk with lots of persistent memory on a flash drive.

Keep in mind, this is just a 32 Gig drive for testing, but nearly all of that is now available to YaCy for storage, (apparently, keeping my fingers crossed.)

This is evidenced from the system status report:

Previously this would only show about 2 Gig total memory and YaCy would immediately complain about low disk space and shut down crawling and other services.

Tom,

It must be exactly as I wrote (all lower case):

persistence=/opt/yacy

If you want to set persistence also for the /home/guest folder then please use:

persistence=full /opt/yacy

‘persistence=full’ works the same as ‘persistence=/home/guest’.

I did have persistence=/opt/yacy but that did not seem to work. Well there was maybe 1 Gig.

I changed that to persistence=full /opt/yacy

There is a space between full and /opt/YaCy

That seems to have opened up the entire drive. There may be other settings that could work, but not to my knowledge, so far.

I think the persistence file could also be set anywhere, even a second USB drive, leaving the boot drive entirely read only. But, that is an experiment for another day.

Take note also:

Hello Tom,

“How long will this link remain active”

Few weeks for sure.

“is it permissible to share the link or post it to the YaCy forum for the developers there who might also be interested?”

Yes

So: Download link:

http://porteus-kiosk.org/custom/421-Porteus-Kiosk-5.1.0-x86_64-yacy.iso

This will, presumably, not be available indefinitely

It is not a ready-to-go kiosk but rather a kiosk configuration and ISO burning utility that has YaCy pre-installed.

During configuration, the important thing is also to set ./startYACY.sh

I have yet to set up and burn the server to get the kiosk and server interoperating.

Another thing I found is that once a usb drive is burned with Porteus, It is not a straightforward matter to simply run the configuration utility and repeat the process, if you want to make any changes or try different settings. Windows cannot reformat the disk and the kiosk configuration iso cannot overwrite the files. Some other utility must be used to reformat the drive before it can be used over.

I found that easy2boot will install on a previously used disk and overwrite whatever is on it, then, if necessary it can be reformatted. Probably there are other disk utilities that could work, but easy2boot is one I had available that seems to do the job of bringing the disk back into useable condition.

Now to test out how well the persistence actually works! Getting YaCy to recognize the available disk space was a challenge.

Using MX Linux, There was persistent memory, but limited to, I think about 10 Gig, regardless of the size of the actual drive.

On Porteus, it appears YaCy is able to see and utilize the entire drive.

I picked up several new USB drives to test this out. The largest is 256 Gig

If that works, I may try moving up to 1 or 2 terabyte, but such drives are not readily available here locally. I would have to order some.

As yet, I have not set up an admin password, but I may do so now, mostly so I can log into admin from my phone, or elsewhere.

I’m also working on installing YaCy on a web page, so I can modify the “splash page”

If anyone else wants to play with this setup, and if the link above stops working, I can upload the file to my own server or something .

Tom_Booth · 31 December 2020 01:55

During installation, the field for entering commands was already populated with something that was highlighted. I took this to be “examples” of how commands should be entered, but now I’m wondering if they were essential Porteus Kiosk setup commands that should not have been deleted?

In the field it says:

dunstify -u low ‘command test’

Whatever it is it was deleted last install, because it was highlighted and got overwritten when typing in ./startYACY.sh which I’m assuming was necessary, as it is impossible to run commands once the kiosk is started as far as I know, which isn’t very far.

Some info here seems to suggest this is just an example intended for deletion:

https://porteus-kiosk.org/parameters.html#run_command

Here I go again.

Tom_Booth · 3 January 2021 06:31

Sorry, somehow I missed this.

They might not. But it would be there, with some “splash page” to look at.

Why do people pay for advertising on diner paper place mats?

Why do they have TV’s in laundromats? Magazines in waiting rooms?

When people are sitting around with nothing better to do, just waiting to be served a meal, waiting for the clothes to dry, waiting for an appointment, if there is a tablet/kiosk displaying anything remotely interesting they may take a look out of curiosity.

Also, the content would be something new and unusual that could capture attention. And mostly it would be oriented towards the needs and interests of the local community. The local community will be involved in creating the content. More a local community communications hub.

It would be just a website that they could also access through their phone, but the kiosk would also serve as a kind of community bulletin board, such as are found in various locations where people can post public announcements, business cards, advertise rooms for rent, put up pictures of lost pets etc.

These kind of things do not automatically pop up on people’s phones, they would have to know about it and navigate to it, whereas a kiosk can be set to a specific page, be made to rotate through various pages.

Tom_Booth · 5 January 2021 03:02

I got frustrated with the difficulty of trying to set up a splash page / kiosk / website and do text editing, control panel access etc. etc. on a crippled browser / OS.

Also, I don’t know how long it might take me to learn or get used to the porteus package management.

I tried SLAX as an OS that is designed to run on a USB drive with persistence.

Technically I guess, YaCy on a Flash drive and YaCy on a Kiosk are two different topics, or two different projects, but I guess I’m killing two birds with one stone.

The links above are working again. This time I set an admin password. All seems to be running well on SLAX, so far.

Screenshots taken from the YaCy-SLAX install.

The normal YaCy installation procedure worked on SLAX via the terminal, ecept it was not necessary to type sudo, for whatever reason i dont know.

apt-get update
dpkg --configure -a
apt-get install -y openjdk-8-jre-headless
wget https://download.yacy.net/yacy_v1.924_20201214_10042.tar.gz
tar xfz yacy_v1.924_20201214_10042.tar.gz
cd yacy
./startYACY.sh

Did some minor customization to the interface:

http://yacy-kiosk.calypso53.com/index.html

all running from a USB drive.

I still need to reboot to see if persistent storage is actually working properly.