Promnesia

• A [[tool]]
  • or perhaps rather a [[collection]] of [[tools]]
  • [[by]] [[karlicoss]]
  • [[open source]]
    • [[git]] https://github.com/karlicoss/promnesia
    • [[firefox]] https://addons.mozilla.org/en-US/firefox/addon/promnesia/
      • [[go]] [[firefox]]
    • [[backend]] pip3 install --user promnesia bs4 lxml mistletoe logzero
    • see also [[promnesia howto]]
  • [[beepb00p]] https://beepb00p.xyz/promnesia.html#what_i_want
    • nice [[exobrain]].

Promensia & HPI

• Remote ssh-tunnel to a promnesia.sqlite VM with [[user-systemd]] listening port to spawn [[ssh]] (see #systemd-user) DONE:
  • reported @ SO for systemctl --user
• promnesia-PR#211 for partial updates: CLI to update DB (fix karlicoss/promnesia#20 for real time indexing)
• PR: Logging INFO by default & configure logging by-file
• automate extraction & indexing with cron jobs
• Integrate Polar Bookshelf by downloading everything in promnesia DB!
• HPI/promnesia extractors PRs:
  • promnesia-PR#204 for Viber source:
    • finalize promnesia-PR#208
    • spread into HPI: But telegram source is also promnesia-only!?
  • HPI/promnesia PR: Signal extractor:
    • export code taken from carderne/signal-export
    • Allow exporting a backup from Signal Desktop, Support creating an archive of messages and media (#1013) and Error: Encrypted export/import is disabled (#2516)
    • fix signal when `http_only``
  • HPI/promnesia PR: Disqus extractor
    • Requested GDPR disqus-archive:
      • Archive came back quasi-empty! Reported, and awaiting reply.
      • support issue
      • notification email
  • Amazon extractor (e.g. books whish-list)
  • Zulip extractor
  • Wikipedia extractor
  • GitHub GDPR source?
• harvest from stexport: OAuth fail!
  • Throttle-limited for 24hours(!), reported in karlicoss/stexport#1, fixed.
• harvest Chrome & Firefox history
• Promnesia-PR to document all the above (see also karlicoss/promnesia#15)
• HPI-BUGs: x2 bugs to HPI & pockexport [[pkdb]] tools
• HPI-PR: fill-in missing URLs from retweets in Twitter-archive (proposed in karlicoss/promnesia#193), based on twitter-entities understanding (originally filed karlicoss/promnesia#193)
• Use desktop pop-up notifications scheme to link from collected promnesia visits.

Table of Contents

• [* data sources](#dtsrcs TIDDLYLINK)
  • [[social networks]]
  • [[hangouts (from google takeout)]]
  • [[Local browser database]]
  • [[Google takeout]] [[takeout]]
    • [[investigation what data is actually contributing to takeout]]
    • [[youtube watch history from takeout]] [[youtube]]
  • [[use likes!]] [[twitter]] [[hpi]]
  • [[Slide database – visited posts? Same for hn]] [[reddit]] [[hpi]]
    • [2020-04-04] right, SEEN database got some stuff /data/data/me.ccrama.redditslide/SEEN
  • [[use my.pinboard]] [[hpi]] [[pinboard]]
  • [[org context – need to strip away todo state changes…]] [[org]]
  • [2020-01-25] http://www.lord-enki.net/links.html import enki's wiki? [[exobrain]]
  • [[android resquetime database?]] [[hpi]] [[rescuetime]]
  • [[do I need to process org archive?]] [[org]]
  • [[android search history? not sure if worth trouble…]] [[takeout]]
  • [2020-09-05] hmm, firefox on android has 'combined' view?? [[firefox]]
  • [[merge all of takeout histories? maybe as long it appears in both]] [[takeout]] [[hpi]]
  • [[wow, there is more stuff in takeouts]] [[takeout]]
  • [[hmm, youtube 'watch-history' is ticking. but Youtube/MyActivity is fully backed up???]] [[youtube]] [[takeout]]
  • [2020-05-01] plus.maths.org |
  • [[google takeout – parse json instead??]] [[hpi]] [[takeout]]
  • [[think how should archives (zip/gz/etc) be handled…]]
  • [[Guess time based on git blame]]
  • [[multiple twitter accounts?]] [[twitter]]
  • [[deliveroo orders]]
  • [[stackexchange? reuse votes and favorites]]
  • [[stackexchange gdpr has lots of visit data]]
    • [2020-12-04] although it's over the whole domain, so not super useful
  • [[right, I am not using chrome anymore, which means takeout history is less and less useful for me]] [[promnesia]] [[takeout]]
  • [[use twidump as extra source?!]] [[promnesia]]
  • [[figuring out history from window titles?]] [[promnesia]] [[arbtt]]
  • [[extract 'regular' links from markdown]] [[promnesia]]
• [* ideas](#ds TIDDLYLINK)
  • [[automatic interface]] [[hpi]] [[promnesia]]
  • [[would be interesting to unify]] [[webarchive]] [[promnesia]]
• [* extension issues/improvements](#xtnsnsssmprvmnts TIDDLYLINK)
  • [[clicking on # on a github page causes reloading highlights…]]
    • [2020-11-19] eh. on the one hand, what else it should do? fragment could be anything
  • [[improving highlights]] [[annotation]]
  • [[extension: ok, highlight by source color in 'show visited' definitely makes sense!]] [[promnesia]]
  • [[performance: pagination should be configurable, i.e. if you run locally you might not need it at all]]
  • [[extract canonical from page url as well? not sure how to make it more friendly and responsive…]] [[cannon]]
    • [2019-08-31] document.querySelector("link[rel='canonical']").href;
    • [2019-08-31] Determining canonical url
    • [2019-09-03] eh, it's a bit tricky because requires extra content script execution and merging. And in 99.99% cases it's gonna be same as canonified url?
    • [2020-05-01] maybe use it first? only then fallback on default algo. could make it configurable?
  • [[trying to debug unresponsiveness after idling]] [[promnesia]]
  • [2020-11-18] All | Search powered by Algolia [[promnesia]]
  • [2020-05-25] tabs.onActivated - Mozilla | MDN [[promnesia]]
  • [[try loading stuff earlier, while URL updates]] [[promnesia]]
  • [[bug: in chrome]] [[promnesia]]
  • [[bug: icon disappears on ctrl-t/close , but manual tab swtich with mouse is fine??]] [[promnesia]]
  • [2020-05-01] doesn't work with readability? Quantum Diaries [[promnesia]]
  • [[bug: chrome: callbacks stop triggering after I reload the extension manually. ugh. Looks like due to the same callback hacking in background.js]] [[promnesia]]
    • [2019-09-09] right, I guess that's because first page that's opened on chrome is options_page…
  • [[need to defensify everything]] [[toblog]] [[webext]]
  • [[Expose db from the server??]] [[malleable]]
  • [2020-05-11] Alexander Flores on Twitter: "@karlicoss having a pane like that but scroll synced to content in article would be ideal for me" / Twitter
  • [[usability: perhaps easier with overlay in the middle of screen? then there are no issues with displaying…]]
  • [[usability: expand all/collapse all]]
  • [[usability: display summary of available contexts on top?]]
  • [[display some sort of 'collapsed' dynamic summary with tags and timestamps only? maybe on the side, and you 'zoom' into it as you scroll]]
  • [[bug: on 'cancel' after blacklist, still shows notification that it added]] [[promnesia]]
  • [[ui: split in tabs: annotations + plain visits? maybe even split by provider?]]
  • [[ui: Sidebar]]
    • [2019-08-01] eh, ended up implementing mine for now…
    • [2019-07-07] jeremychurch/FixedContent.js: A mobile-friendly, jQuery plugin that persists sidebar content, nav, etc.
    • [2019-07-07] arkon/ng-sidebar: Angular sidebar component.
  • [2019-04-19] ui: Designing a Personal Knowledgebase – A Curious Mix [[promnesia]] [[org]]
    • [2019-12-26] i guess it was referring to unhighlighted links etc?
  • [[bug: show visited mess with grasp?? (try selecting over these marks and capturing text)]] [[promnesia]]
  • [[excludelist: www.site.com vs site.com]] [[promnesia]]
• [* github issues](#gthbsss TIDDLYLINK)
  • [[extension: would be cool to have an option to just 'find' empty space on screen to overlay… seems hard though]] [[ml]] [[think]]
    • [2019-07-20] more generally, non-invasive? and not sure if ml is necessary for that at all…
  • [2020-09-13] Cannabidiol - PsychonautWiki
  • [[multiple processes]]
    • [[use multiple threads for indexing…]]
    • [2020-11-08] ugh fuck, def useful for indexing code..
  • [[load more if page is scrolled? gonna be tricky…]]
• [* indexer: bugs/issues](#ndxrbgssss TIDDLYLINK)
  • [[hmm. if we switch db mode to 'update', then schema changes would break promnesia…]] [[promnesia]]
• [[hmm, what if people used it with older orgparse? (suggest to update in release notes)]]
• [[personal style settings]]
• [2020-11-22] extension: twitter is pretty slow & logs are spammy Home / Twitter
  • [2020-11-23] ugh, so also impacts 'show this thread'
• [2020-06-07] inkandswitch/ksp-browser: Connect the things you already know in your browser. [[pkm]]
  • [2020-09-11] hmm, need to outreach them?
• [[different notions of 'visited']] [[think]]
• [[figure out what could possibly be the most interesting source of links for people? I'd imagine whatsapp?]]
  • [2019-07-15] reddit definitely. Slightly harder to motivate backups [[reddit]]
  • [2019-07-15] could integrate with my reddit backup script then? reindexing would trigger backup; seems kinda ok
  • [2019-08-31] pocket perhaps?
  • [2019-08-31] actually, pinboard; keep one backup in repository to test the extractor or demonstrate json extractor? [[pinboard]]
• [2019-12-15] Andy Matuschak on Twitter: "@LiquidTextCorp I'm very excited to try!" [[motivation]]
• [2020-03-22] It would be also great if it would work backwards aswell, eg.: "the link I found on twitter last week"
  • [2020-05-28] I mean it will once I integrate with the database properly…
• [2020-05-03] (7) Spencer Chang on Twitter: "@jborichevskiy @Twitter @Wikipedia @hypothes_is @RoamResearch Love this have been thinking in the same space where the content you consume is linked in-place to the content you (or the people you think are important) curate. Looking forward to seeing where this goes!" / Twitter [[ui]]
• [2020-05-06] WorldBrain.io on Twitter: "We are starting the development of the Memex Reader, an offline-first Pocket-style reader for desktop and mobile + mobile annotations. https://t.co/5lUpNSImFe Is anyone interested in collaborating/contributing to this? We need some more (wo)man power. ❤️" / Twitter [[worldbrain]]
• [2020-06-22] WorldBrain.io on Twitter: "We're working on sharing/collaboration features of Memex and would love to have your input! Watch a quick rundown on some early mockups: https://t.co/brncMElHFr We'd love to have you for a 30-min call to get a grip on your use cases: https://t.co/AT52JajznO" / Twitter [[worldbrain]] [[social]]
• [2020-06-17] TrailHub on Twitter: "https://t.co/do5RBGjjk4 Ready to give StorexHub a whirl! Take @hypothes_is annotations, and bookmark the pages in memex so that they will show up as a liked page in memex and be indexed Add Page notes to Memex that link to hypothesis annotations https://t.co/SUk55b3ADn https://t.co/QsybMSwHxo" / Twitter
• [2020-03-21] westoncb/mymex
• [[demo: on malleable systems collective, with screenshot from the chat itself?]] [[promnesia]] [[social]]
• [[usecase: reading hackernews 'new']]
• [[demo: 'mark visited' on the hackernews top?]]
• [2020-02-11] mek.fyi | Home
  • [2020-11-19] tweet at Mek after releasing new version?
• [[demo: post again on /r/orgmode? with demos of highlights/translusion/backreferences?]]
• [2020-11-19] demo: good demo for mark visited Subscriptions - YouTube
• [2020-11-17] usecase: New Links | Hacker News
• [[ugh, something's not working with release build in firefox??]]
• [[discuss on malleable systems how hard was it to modify youtube's website and inject annotations etc]] [[promnesia]] [[outbox]]
  • [[could also discuss with Oliver from worldbrain memex?]] [[worldbrain]]
• [[demo: tweet HN new reading when I encounter some user I know]]
• [[publish: release on lobsters? seems ok]]
• [2020-11-21] control logging in content scripts
• [2019-07-08] testing: unitest/History at master · scriptmasters/unitest [[hpi]] [[jdoe]]
• [2019-11-22] highlight: https://news.ycombinator.com/item?id=21403294
  • [2019-12-27] I guess if I show it on HN, better get it right
• [[test: for the end2end test, run it against a database indexed with an older (PIP) version]]
  • [2020-11-19] actually not super important; in most cases database is overwritten
• [[docs: make sure it's possible to run merely by creading config, then index, then serve]]
• [[Please visit https://editorjs.io/ to view all documentation articles. Base concepts]]
• [[for demo, could just anonymize people? that would solve 99% of it]] [[promnesia]] [[demo]]
• [2020-05-11] deen-chan on Twitter: "@ajflores1604 @karlicoss @jborichevskiy I've implemented this (augmented sidebar) and it's quite hard to come up with something that works well for all sites. You quickly run into sites which don't reflow, assuming a certain layout. But that was years ago. Responsive design is a lot more common these days so might work" / Twitter
• [2020-07-05] Guide to using filemagic — filemagic 1.6 documentation
• [2020-05-20] I remember using this software last time, it is wayyyy~ too buggy, it stalls, cr… | Hacker News [[promnesia]] [[worldbrain]]
• [[fucking hell. figure out how to fix version exposed during the adhoc install….]]
• [2019-12-12] demo: Digital Tools I Wish Existed :: Up and to the Right — Jonathan Borichevskiy
  • [2020-03-28] maybe make a screenshot out of it?
• [2020-05-20] Introduce Yourself - Thinking Tools [[social]] [[publish]] [[promnesia]]
• [2020-04-29] integrate in Memex? [[worldbrain]]
• [[post automatic demo gifs]] [[publish]]
• [[doc: minimal example to import something in promnesia, with crontab examples etc]] [[promnesia]]
• [maybe it should be emacs:// ? not sure](#mybtshldbmcsntsr TIDDLYLINK) [[mimemacs]] [[promnesia]]
  • [2020-05-21] or editor:// ??
  • [2020-05-29] ok, so it seems that emacs:/// works and is backwards compatible, which is kinda good news
    • [2020-05-30] right. so I guess I know what to do
  • [2020-05-29] could add to doctor?
  • [2020-11-01] maybe mimes should be configured & tested in the frontend? could add to the extension troubleshooting page
• [2019-12-08] motivation:
• [[demo: of 'mark visited']] [[toblog]] [[promnesia]]
  • [2020-05-16] hmm, make sure visited marker/css is tweakable? I suppose it needs to be in the 'main', sidebar section. confusing…
    • [2020-05-16] post about it – why don't we abuse a human brain which is much better at pattern recognition than computers (so far)
• [[publish packed zips in releases; assemble automatically on ci]] [[ci]] [[webext]] [[promnesia]]
• [[remove 'duration' from the database? hopefully no one uses it yet…]]
  • [2020-05-14] actually I might need it later.. if I populate them from #arbtt
• [2020-01-13] usecase: l3kn/org-fc: Spaced Repetition System for Emacs org-mode
• [2019-11-01] ui: Shtetl-Optimized » 2016 » April
  • [2019-11-04] allow to be configured via CSS
    • [2019-11-04] how to automate it actually?
• [2019-11-09] bug: Introduction - Everything I know
  • [2019-11-12] eh. fair enough, it's changing URL. Not sure if there is much that can be done?
    • [2019-12-25] except if cannon works clientside. Ugh!
  • [2019-12-07] can be solved temporarily with proper blacklisting
• [[be less annoying about errors.. not sure how though]] [[errors]]
  • [2019-08-31] just implement different options for notifications?
  • [2019-09-01] quick option do disable/enable notifications; later find some generic component to snooze them?
    • [2019-09-09] yeah, I guess they'd be ok on sidebar? Basically, make sure sidebar always slides since it's convenient for settings, access to search etc.
• [[Def need way to make it less spammy on network errors on phone. System wide notification is too much]] [[errors]]
• [[test going to different url in same tab; not sure how it would work…]]
• [[support stuff without tz? or at least warn and maybe instert as utc. or just pass forward local tz that's ok for most people]]
• [2019-08-29] mozilla/webextension-polyfill: A lightweight polyfill library for Promise-based WebExtension APIs in Chrome [[webext]]
• [2019-09-10] handle when pages aren't available better
  • [2019-09-23] need to implement at least popup
• [2019-09-05] lukeed/tinydate: A tiny (349B) reusable date formatter. Extremely fast! [[datetime]] [[js]]
• [2019-07-14] Surprisingly Turing-Complete - Gwern.net
• [[I guess sort of killer feature would be 'hierarchical' visits (e.g. facebook pages, reddit comments, github. etc. for reddit will require some manual rules against canonified urls)]]
  • [2019-07-22] search kind of contributes towards that
• [[stuff in database seems to be unnormalised (e.g. case). need some sort of checker …]] [[cannon]] [[togithub]]
  • [2019-06-16] hmm, do that I guess sqlite3 visits.sqlite "select norm_url,tag from visits where norm_url LIKE '%usg%'"
• [[useful to populate from old backups (e.g pinboard) in case I delete articles/bookmarks]] [[motivation]] [[promnesia]]
• [[filter links]]
• [[warn about anomally long histories?]]
• [[would be nice to process links/pieces of information that I merely seen!]]
• [[like hypothesis, but process the page server side and highlight against everything in the knowledge base]] [[hypothesis]] [[pkm]] [[promnesia]]
• [[would be nice to know how I got onto the page…]] [[promnesia]] [[motivation]]
  • [2019-06-02] use chrome from_visit?
  • [2021-01-20] search basically does this
• [[need some thing which normalises and merges messages? or maybe just disregard duplicate links from old and new backups…]] [[backup]] [[telegram]] [[promnesia]]
• [[could analyse 'how did I get there' from my web archives]] [[archivebox]]
  • [2019-07-31] I guess I meant from plaintext search
• [[publish: /r/dataisbeautiful post my visited urls stats?]]
• [[Tweet from 𝔊𝔴𝔢𝔯𝔫 (@gwern), at Mar 10, 23:59]] [[promnesia]] [[motivation]]
• [2019-04-15] Pinboard: bookmarks for tswaterman tagged 'math'
• [[locators need short name along the full path]]
• [[hmm, def looks like chrome is syncing bits of local database as well. ugh]]
• [[polar got OK design for sidebar]] [[polar]] [[promnesia]] [[inspiration]]
• [[custom search engine?]] [[search]]
• [['came from' is just a context. Would cover 99% of cases]]
• [[tags are pretty useful…]]
• [[Works really well in conjunction with axol (both dots and contexts)]] [[axol]] [[promnesia]] [[motivation]]
• [[showvisited: show dots on selection? useful so you don't need to send too many queries. or anchors like surfingkeys?]]
• [[display error in sidebar?]] [[errors]] [[ui]]
  • [2019-07-14] could be useful if we can extract URL; but encountered some issues while extracting context etc; could emit both visit and error
• [[not sure, maybe I should just use native async/await? Most mobile and desktop browsers support it]]
  • [2019-07-15] generally figure out how much can I get away with using modern JS as opposed to webpacked
• [[go through existing urls and try to normalise them?]] [[webarchive]]
• [[ught https://github.com/brookhong/Surfingkeys/blob/57fccbbeeb60ee2be0d2d60cfc50bd3aca3b0436/background.js#L1094]] [[webext]]
  • [2019-07-07] shit. ok, so chrome experiment confirms that it just doesn't work for extensions.
  • [2019-07-07] shit. do I need bookmarklet or what???
• [2019-07-08] Search · filename:places.sqlite https://github.com/search?p=2&q=filename%3Aplaces.sqlite&type=Code
• [2019-07-14] mitchellkrogza/Ultimate.Hosts.Blacklist: The Ultimate Unified Hosts file for protecting your network, computer, smartphones and Wi-Fi devices against millions of bad web sites. Protect your children and family from gaining access to bad web sites and protect your devices and pc from being infected with Malware or Ransomware.
• [[merge results together? e.g. search for inmotionmagazine]]
• [2019-03-08] motivation: Nikita Lisitsa on Twitter: "@_bravit У меня так на stackoverflow / math.stackexchange бывает. Ищу ответ на вопрос, нахожу, читаю, ставлю лайк - "вы не можете поставить лайк своему собственному посту"." / Twitter [[promnesia]]
• [2019-07-08] Does Firefox ship with a new default for "history expires after…"? - Super User
  • [2019-07-08] Does Firefox ship with a new default for "history expires after…"? - Super User
• [[for mimemacs, could probably support vim primarily (since all users will have it), it could even fallback]] [[mimemacs]]
• [2019-07-23] demo: Best Mangal Bar & Kitchen delivery from Farringdon - Order with Deliveroo
• [[rss blogs, easy to see what clicked and what hasn't]] [[rss]]
• [[should show visual indication on 'no results']]
• [[when I merge together browser activity etc, I need to assert on some historic entries to make sure timestamps match properly]]
• [[would be too annoying to deal with transitions I guess, considering that it's not needed too often. Just rely on jumping into timeline?]]
  • [2019-08-01] ??
• [[open in new tab from search?]]
• [[how did I get here – perhaps shouldn't mix context and visits??]]
• [[yeah, searches useful to have in db for context; even though they'd almost never match and normalise]]
• [[yeah, show url split in parts; then when I click one of the parts in queries more data?]]
• [[direct link to css from the extension?]]
  • [2020-04-28] I guess this is possible judging by greasemonkey
• [[could expand the grouped things on grey area press?]] [[promnesia]] [[togithub]]
• [[maybe use two locators? one for linking and one for debugging]]
  • [2019-12-26] eh?
• [[moz-extension://b966cab9-aded-44e8-b116-a900ab825442/search.html?timestamp=1561844122.151]]
• [hmmm author_url could be useful?](#hmmmthrrlcldbsfl TIDDLYLINK) [[hpi]] [[reddit]]
• [[bug: on android doesn't respond well on filter clicks, and overall looks a bit weird..]]
• [shows notification on navigation within page when clicking anchors http://super-memory.com/articles/sleep.htm#biphasic_sleep ,perhaps shouldn't do that…](#shwsntfctnnnvgtnwthnpgwhntmbphscslpprhpsshldntdtht TIDDLYLINK)
• [[enable back options to always show dots on visited urls?]]
• [2019-08-31] uBlock/manifest.json at 6c34b3c3c96756b6db7ff2f3a0394472d81cde3e · gorhill/uBlock [[webext]]
• [[allow per-device settings?]] [[webext]]
• [[bug: on restoring tab, shows notification twice. weird]]
  • [2019-09-22] doesn't seem to do with hacky injector backed initialization either. odd
  • [2019-09-22] also only happens in firefox, apparently…
• [[https://slatestarcodex.com/2014/03/17/what-universal-human-experiences-are-you-missing-without-realizing-it/]]
• [2019-07-30] hmm it triggers on clicking anchors. not sure it's a good idea? also add to end2end tests
• [[local sync vs cloud? Not sure how to handle this properly; maybe have different profiles?]]
• [2019-09-09] test: fabianonline/telegram_backup: Java app to download all your telegram data.
• [[fails on firefox.com domain; fair enough, but perhaps need to blacklist it?]]
• [missing host permission – very consistently reproduces on test_visits if you close/reopen tab](#mssnghstprmssnvrycnsstntlyrprdcsntstvstsfyclsrpntb TIDDLYLINK)
• [[shit, tag map didn't work with context visits]]
• [[have 'primary' settings profile and just think about rest later]] [[webext]]
• [[on android, maybe should only request on sidebard click? doesn't display icon anyway… then basically it'd solve systemwide notification thing]] [[promnesia]] [[togithub]]
• [[:invalid selector and input validation]] [[blog]] [[webext]]
• [2019-10-19] Re: [fregante/webext-options-sync] Race condition in OptionsSync constructor
• [[Highlight blacklisted and paywalled websites]]
  • [2020-11-19] with mark visited it's much easier now.. could even have a special source and apply special style to them?
• [2019-11-04] performance: vinta/awesome-python: A curated list of awesome Python frameworks, libraries, software and resources
  • [2020-11-14] hmm, now it's fine but it highlights a bit too much info? not sure what to do about the fragment normalisation…
• [2019-11-04] promnesia/hypothesis.py at master · karlicoss/promnesia
• [2019-11-09] On todo lists | beepb00p
• [[Tweet from Will Manidis (@WillManidis), at Nov 26, 15:44]]
  • [2019-12-26] ugh, tweet is gone??
• [[ui, phone: don't really have to tweak body? as it's easy to hide/show sidebar]] [[promnesia]] [[togithub]]
• [[I don't like some twitter account I'm following: they are whining too much or whatever]] [[twitter]] [[promnesia]] [[motivation]]
• [[Build client only version as demonstration?]]
• [[query database]]
• [[org-mode: comments from LOGBOOK might be useful..]] [[org]]
• [[for search might be good to display original link??]]
• [2019-12-15] bug: Теория вычислимости — Викиконспекты
• [[Could map different people onto the same 'entity'?]]
• [[after loading browser with previously open tabs and then closing them; shows 'host permission' warning]]
• [[should allow running against live database? really why not, could be nice for exploring]]
• [[Maybe git tracked server config is not so bad]]
• [[Use hub/click? And maybe docker to isolate]] [[docker]] [[python]]
• [[Would be rad if it was possible to simply run it against GitHub repo straight from ui]]
• [2020-01-25] Robot&AIWorld on Twitter: "Here's more footage of MIT's Mini Cheetahs cavorting, frolicking, back-flipping, playing soccer and generally acting fun and cute, courtesy of the Biomimetic Robotics Lab @MITMechE https://t.co/8ZQzDvCDVW" / Twitter
• [[Allow raw settings backup]] [[webext]]
  • [2021-01-20] this should really be easier…
• [[shit, google docs trigger a lot of reloads.. really should debounce]]
• [[duplicating search page – fucking hell. still happens on firefox…]]
• [[chronic versioning?]] [[project]]
• [[ugh. subdomain needs to be banned..]]
• [2019-12-27] Adventures in WhatsApp DB — extracting messages from backups (with code examples) [[promnesia]] [[whatsapp]]
• [2019-12-30] What am I meditating for? In Pursuit of A Definition of Meditation - Mark Koester
• [[warning about module size…]]
• [[not sure about tabs vs activeTab permission]] [[webext]]
  • [2020-02-15] maybe instead I could simply check last requested url on tab switch?
• [[ok, I guess I could make "tabs" an optional permission; by default only request stuff via click on a popup (that requires 'activeTab'?)]]
• [[shit, request on every tab switch is not good…]]
• [2019-08-10] right, chrome doesn't support android extensions. could use bookmarklet or something?? https://stackoverflow.com/a/10606887/706389
• [[as a docker app?]]
• [[go through code and create issues?]]
• [[eh, web indexer doesn't really work for relative links…]]
• [[combine with eye tracking, then could know how much time was spent reading something…]]
• [[would be nice to get stuff collected by axol to promnesia database]] [[axol]] [[promnesia]]
• [[shit, 2mb per single page (10K visits) sucks…. wonder if should use some different protocol? Or gzip defeats the purpose?]]
• [2019-12-05] Erik Torenberg (@eriktorenberg) / Twitter [[motivation]] [[demo]]
• [[use dev webdriver??]]
• [[demonstrate on that place with great bread]] [[promnesia]] [[demo]]
• [2020-01-25] hyperhype/hyperscript: Create HyperText with JavaScript.
• [2020-03-30] Troubleshoot extensions, themes and hardware acceleration issues to solve common Firefox problems | Firefox Help [[webext]]
• [[maybe add sidebar menu on mobile only? anyway, need to add blacklist]]
• [['mark visited' works really nice on the phone]]
• [[doesn't work in private windows (I assume because of the getActiveTab thing)]]
• [[common usecase – I follow someone, then I'm annoyed by their tweets and try to figure out why had I followed them in the first place?]]
• [[Maybe we just need a proxy that logs literally all of http requests.. That way history could be agnostic]]
• [[shit, webdriver only took screenshot of half of the frame….]]
• [[not compatible with gh: style links…]]
• [2020-04-07] usecase: Idle Words https://idlewords.com/
• [[could test on twint??]] [[twitter]]
• [[usability: keyboard navigation]] [[webext]] [[malleable]]
  • [2019-08-01] ugh. surfingkeys is capable of working on my iframe, but can't switch
  • [2019-08-26] at least write about it in readme/faq
• [[could draw a small 'visits/contexts' tree? would be fun..]]
• [[maybe backend could update rules now and then..]] [[cannon]]
• [[Breakdown of contexts by parts of virtual url in the sidebar]]
• [[attemt to set up autorealoding]] [[promnesia]] [[webext]]
• [[Write a post demonstating which bits should not be part of extension]] [[malleable]] [[promnesia]] [[toblog]]
• [[Hmm. Sending fragment can be unsafe?]]
• [[use arbtt to match the browsing history? that would be fun…]] [[arbtt]] [[hpi]]
• [[readme: probably, better to install it via pipx? so HPI etc are easier to upgrade?]] [[python]] [[promnesia]]
• [2019-11-21] raxod502/mercury: Emacs interface to Facebook Messenger [[facebook]] [[hpi]]
  • [2019-12-02] could be useful…
• [2019-09-22] 1397667 - "No matching message handler" error when tabs.update().then(tabs.executeScript()) [[webext]]
• [2019-09-22] 1290016 - tabs.executeScript in webRequest.onCompleted gives "Unchecked lastError value: Error: No matching message handler" [[webext]]
• [[visual web scraper? geoffrey litt mentioned generalised nocode stuff (Pavel's question)]]
• [2020-05-07] eslint-plugin-no-unsafe-innerhtml - npm [[webext]]
• [[be more informative; show full history or at least last visit and potentially sources (e.g. hypothesis)]]
• [[attempt at agnostic url extraction (sha ae889fa0fb99f683cd1ba6192a3b55d11a481558)]] [[reddit]] [[promnesia]] [[hpi]]
• [[demonstrate jumping to the message on android? ironically it works better on phone]] [[promnesia]] [[demo]]
• [[time spent? only for chrome apparently]] [[promnesia]] [[demo]]
• [[Annotating post]] [[worldbrain]]
• [2020-05-20] Memex/manifest.json at develop · WorldBrain/Memex [[project]]
• [2020-05-23] Programming Inside a Container | Hacker News [[docker]] [[hpi]] [[promnesia]]
• [[what if I could apply :visited state for links that I marked as seen?]]
• [[settings – add visual cues for dots/ visits/contexts/sources]]
• [2020-05-13] Native Messaging - Google Chrome [[webext]] [[promnesia]]
• [2020-05-27] Show HN: Obsidian – A knowledge base that works on local Markdown files | Hacker News
  • [2021-01-20] some people apparently already used against obsidian db?
• [[Lesson]] [[promnesia]] [[publish]] [[think]]
• [[Hmm, maybe do not highlight stuff from archived org files/deleted instapaper/etc?]]
• [[Introduction to Marionette — Firefox Source Tree Docs 78.0a1 documentation]] [[webext]] [[promnesia]]
• [[tried disabling logic for single background page – doesn't help even with persistent background..]]
• [2019-05-24] useful to have links just added to instapaper to know that you are planning to read them there
• [2020-04-29] TypeError: can't access dead object - JavaScript | MDN
• [2020-05-05] type alias to union is invalid in runtime context · Issue #5354 · python/mypy
• [[docs: mention that I'll be posting about breaking changes in releases]]
  • [2020-05-29] even better to do this in commits and simply reference stuff in releases? or, sync releases with a file in repository
• [2020-06-03] Using Service Workers - Web APIs | MDN [[webext]]
• [2020-06-03] highlights: Quora Answers by David Pearce (2015 - 2020) : transhumanism with a human face
  • [2020-11-14] hmm, performance OK, but it throws ruandom erros now??
• [[They record where you come from, what pages you visit, how long you stay on each, where you click and where you go next.]]
• [2019-07-28] Firefox doesn't recognize telegramdesktop's tg link. /r/firefox [[telegram]]
• [2020-05-25] Dmitry Bobrov on Twitter: "@karlicoss No idea about the service, but I do have experience with Chrome extensions if that’s what you mean" / Twitter
• [2020-06-11] doc: Share database between machine? · Issue #114 · karlicoss/promnesia
• [[reuse readability extensions for better content highlight?]]
• [[could use both for reddit cumulative state and promnesia browser exports]] [[hpi]] [[reddit]] [[cachew]]
• [[Post archive org highlights]] [[outbox]]
• [[merge mode – could combine public and private data]]
• [2020-05-11] ESLint v7.0.0 Released | Hacker News [[promnesia]] [[webext]]
• [2020-05-25] Korobochka on Twitter: "@karlicoss Via Twitter?) For the last ~6 months I am developing a Chrome extension at work, can take a look as well." / Twitter
• [[version detection – thing in cachew works pretty well? perhaps it might not work without installing, but otherwise should?]]
• [[HPI/promnesia: a whitepaper?]] [[publish]]
• [[ok, could use some design inspiration from ampie]] [[ampie]]
• [2020-09-25] (20) Anyone use Promnesia? | Building a Second Brain
• [[hm, for serving reuse config, but allow to split config bits to make defensive?? not sure..]]
• [[could use pandoc for serving HTML?]]
• [[about realtime indexing (give a demo of entr??)]] [[totweet]] [[promnesia]]
• [[support child visits etc for inbrowser history]]
• [[dynamic JS hook only in dev mode?]]
• [[how to display warnings in the very end? extract from warnings module maybe?]]
• [[using generators to defer UI updates?]] [[toblog]] [[promnesia]] [[js]]
• [[ideally configuration would be kinda like a grid? for every (almost) source and for every function]]
• [[sharing links with surfingkeys? e.g. they could both filter crap links]]
• [[sidebar isn't opening if new tab, then google search (firefox)]]
• [[check blacklist right before querying backend as the last resort?]]
• [[for excludes, 'do not mark this link' and 'do not mark this element'… the latter is gonna be way harder]] [[togithub]] [[promnesia]]
• [[usecase: when I follow someone on github/twitter/etc I'll make a quick note. Next time I'll know why]] [[demo]]
• [2020-11-10] inkandswitch/ksp-browser: Connect the things you already know in your browser.
• [2020-11-19] Essays · Gwern.net
• [2020-11-18] Watch later - YouTube
• [2020-11-17] Merveilles it's pretty dynamic and might not work well with 'mark visited'
• [2020-11-17] Withings body+ vs. Withings body cardio : QuantifiedSelf
• [2020-11-14] What forces layout/reflow. The comprehensive list.
• [2020-11-16] TweetDeck
• [[hmm, could serve the exlcudelist from the backend???]]
• [2020-11-22] The File System Access API: simplifying access to local files [[promnesia]]
• [[visited marks might impact ellipsized content]]
• [[faq: some pages may prevent extensions on working on them]]
  • [2020-11-22] eh, not sure if this is the case with extensions? bookmarklets (like Hypothesis on HN – yeah)
• [[Add donation link, to bastien?]]
• [[if I use sidebar, it will work even on websites that forbid html injections?]] [[togithub]] [[firefox]]
• [[maybe I need something a bit more discrete for contexts notification…]]
• [[shows host error exceptions in firefox on pdfs]] [[promnesia]] [[togithub]]
  • [2019-12-26] could be disabled via blacklisting *.pdf probably?
• [[process archive as well?]] [[promnesia]] [[org]] [[togithub]]
• [[could hide annotations via tags?]] [[promnesia]] [[togithub]]
• [[backlist: tampermonkey?]]
  • [2020-11-16] eh?
• [2020-05-25] iorate/uBlacklist: Blocks specific sites from appearing in Google search results [[webext]] [[promnesia]]
• [[add some throttling for the notification at least]]
  • [2020-02-25] added suppressing..
  • [2020-03-23] maybe, it's better just to turn it off by default…
• [[prefill with domain name?]] [[promnesia]] [[search]]
• [[ublock content picker is awesome… would be nice to find one..]]
• [[blacklisted: allow to force loading (from sidebar)]]
• [[sidebar doesn't work while search is open :( :(]]
• [2020-05-02] Plain text linkifiers comparison (work in progress) (http://jsbench.github.io/#54315d74264c857856c73f18c81278dc)
• [[closing tab before it's finished loading causing error. I guess that's fine and shouldn't be defensified..]]
• [[not sure how to handle desktop notifications vs builtin? I'd prefer to use system as far as we can I guess but they are not easy to fine tune]] [[webext]] [[togithub]]
• [[host permission on closing and reopening tab?]]
• [[right, switching to highlight.js/codejar does make it 110kb (vs 170 kb)]]
• [#hypothesis adds special element <hypothesis-highlight>? wtf??](#hypthssddsspcllmnthypthsshghlghtwtf TIDDLYLINK)
• [2020-03-02] windows 10 - Disable Chrome to ask for confirmation to open external application everytime - Super User [[promnesia]] [[mimemacs]]
• [[ignored entries should be colored as black]] [[promnesia]] [[togithub]]
• [[could merge (e.g. chrome and takeout) both on client and server; not sure which would be best]]
• [replace fixture with tmp_path like in orger?](#rplcfxtrwthtmppthlknrgr TIDDLYLINK)
• [2019-09-15] bug: raw.githubusercontent.com/karlicoss/dotfiles/master/surfingkeys/config.txt
  • [2019-09-22] ugh. works fine in dev firefox and doesn't even have sources on release?
  • [2019-09-23] also seems to be fine in chrome, however doesn't display kkj
• [[process confidences and somehow add this information in visit line]]
• [[show with discarded url get params?]] [[promnesia]] [[togithub]]
  • [2019-06-02] eh??
  • [2019-12-27] I guess I meant ignore normalisation etc
• [[pinboard last seen]]
  • [[I guess needs a pinboard provider?]]
• [[fuck :D google takeout had json formats for activity…]] [[backup]] [[promnesia]] [[takeout]]
• [[use really really old takeout and see what happens]]
• [[ctrl-r doesn't seem to update extension icon for stale tab]]
• [[Motivation for browser history: its just easier to grab than google takeout data. I just need to merge them together i suppose]] [[motivation]]
• [[option to disable dots on the page (permanently or temrporary)]]
• [[Maybe desktop integration?]]
  • [2019-12-26] what?
  • [2021-01-20] like a remembrance agent maybe?
• [[backend: URL extraction via NN?]]
• [[hmm it sort of around when I'm clicking links on the page.. would be interesting to see how that works]]
  • [2019-08-05] eh?
• [[Watch "There is No Algorithm for Truth - with Tom Scott" on YouTube]] [[promnesia]] [[motivation]]
  • [2019-11-21] eh, not sure..
• [[people who followed you?]]
  • [2020-11-20] not sure what I meant here?
• [[wonder if it would help me to figure out what was the video https://www.youtube.com/watch?v=Z-R2xO3phVM]]
• [[fallback in sequence?]]
• [[right, spend a bit debugging sidebar with 10K visits]]
• [[tell about fake systemd?]] [[toblog]]
• [[Would be fun to run the backend on Android..]]
• [[need to fix compare db to allow for small relative changes? at least for pinboard and reddit]]
• [[pydeps src/promnesia]] [[python]] [[hpi]] [[promnesia]]
• [[Hmm wonder if could adapt promnesia for Next browser?]]
• [2020-04-24] usecase: compute-space/compute-space.md at master · jauntywunderkind/compute-space
• [2020-04-24] usecase: westoncb/mymex
• [2020-05-06] Firefox for Mobile now supports NoScript, PrivacyBadger, HTTPS Everywhere | Hacker News
• [2019-01-19] Switching over to communicating via private server
• [2020-06-30] Native messaging is extremely broken and has bad documentation. I never got repl… | Hacker News [[webext]]
• [[browser history: clicked 'forget about this site' for google.com by accident.. wonder if it's gonna erase stuff]]
• [2020-07-06] Linux kernel coders propose inclusive terminology coding guidelines, note: 'Arguments about why people should not be offended do not scale' • The Register
• [2020-11-06] jeanralphaviles/comment_parser: Python module to extract comments from source code files of various types.
• [2020-11-08] target browsers insteadd of Node, remove old react stuff, update eslint babel · karlicoss/promnesia@c8380e0
• [2020-11-16] uBlock/webext.js at 5aca41960373656c79d547b1a983f053e6adb633 · gorhill/uBlock
• [[bug: right, seems that extension disconnects after a timeout in chrome based browsers]]
• [2019-09-01] keepassxc-browser/manifest.json at develop · keepassxreboot/keepassxc-browser [[webext]]
  • [2019-09-08] figure out why is this necessary?
• [[hmm, need to overwrite database in order to handle schema updates? not sure what would be a clean way of doing it…]]
  • [2019-08-07] I guess similar to cachew?
  • [2019-09-03] ok, although haven't tested that behaviour
• [[I really need page notes… e.g. reasons why I removed the bookmark and whether I already visited it..]]
  • [[maybe, set up inotify to update the database?]]
• [[maybe show yellow icon if we only have chrome visits? so in a sense, green is more 'interesting' visits. also, maybe blink with green for a while (and have a timer whether we blinked in the past hour or something)]]
  • [2019-06-01] need automatic generation for differently colored icons…
  • [2019-06-02] distinguish contexts as well
  • [2019-08-08] just needs a better documenting..
• [[share code with js …. ugh, all the projects I found really suck…]] [[ffi]] [[malleable]]
• [[highlight/color certain tags (configure that in settings)]]
  • [2019-06-01] suggest tags? maybe by querying from api, although presumably there wouldn't be too many. make the config YAML
• [[make sure it searches in archive…]]
• [[Use cachew for browser history]] [[promnesia]] [[cachew]]
• [2020-05-27] sshkarupa/url-handlers [[mimemacs]]
• [2019-09-01] vinta/awesome-python: A curated list of awesome Python frameworks, libraries, software and resources
• [[at first, make filters private when I release it]]
  • [2019-10-12] ok, they do save a bit of space for me, e.g. 575Mb vs 376 Mb. I guess keep my private ones for now and keep them empty in config, can resolve later
    • [2019-10-27] fuck I'm so glad I'm keeping notes! Would have never remembered why db size changed otherwise..
• [[reddit link in context]]
• [[would be also useful to highlight visited posts, e.g. here https://www.lesswrong.com/posts/vwqLfDfsHmiavFAGP/the-library-of-scott-alexandria]] [[promnesia]] [[demo]]
• [[attempt to use prismjs]]
• [[wonder if I can embed in a page, as an Iframe?]]
  • [2019-12-26] would be hard because of need to login, etc…
• [2020-02-18] Upload Add-on :: Promnesia :: Add-ons for Firefox
• [2020-10-28] thesephist/histools: A collection of tools for generating data visualizations from browser history data
• [[release: suggest to update the backend]]
• [[need to figure out how to keep error traceback/context..]] [[python]] [[promnesia]] [[errors]]
• [[I failed miserably at reaching out to Worldbrain/Hypothesis to discuss whether we should collaborate :worldbrain:hypothesis]]
• [[How to run as a proxy server? So it would work with any apps]]
• [[Tweet from @posobin]] [[promnesia]] [[inspiration]]
• [2020-11-07] ugh. looked in codemirror src, but unclear how to optimize the size further. it's just big shrug
• [2020-05-06] agnostic extraction – extracting context is pretty much impossible [[promnesia]] [[hpi]]
• [2020-11-01] profiling: so overall it seems that half time spent in parsing org-mode, half in url extraction. meh
  • [2021-01-20] this py-spy thing is very nice [[performance]] [[python]]
• [https://www.freecodecamp.org/news/lossless-web-navigation-spatial-model-37f83438201d/ https://merveilles.town/@aynish/105139837128207627 pull](#swwwfrcdcmprgnwslsslsswbntnsptlmdlfdsmrvllstwnynsh TIDDLYLINK)
• [[email mek]]
• [[mark visited – maybe for ignored links add an attribute/explanation or something]]
• [[what if I add 'fake' history into browser database? wonder if that would work…]]
• [2020-11-26] Nyxt
• [2020-11-27] showvisited: Новые серии / MyShows.me
• [2020-12-14] Gephi - The Open Graph Viz Platform [[timeline]]
  • [2020-12-14] could draw en edge if the visits are close in time?
• [2020-05-17] All Time - Hacker News Top Links
• [2020-01-22] Playing around with Chrome's history https://gist.github.com/dropmeaword/9372cbeb29e8390521c2#chrome [[chrome]]
• [2020-12-05] interop: DIYgod/RSSHub: 🍰 Everything is RSSible [[rss]] [[promnesia]] [[wildcard]]
• [2020-12-06] Org-roam User Manual
• [2020-12-06] Org-roam User Manual [[grasp]] [[promnesia]]
• [2020-12-06] Newsletter 10 - Neovim v0.4.4 - Neovim
• [2020-12-13] showvisited: Mile End Climbing Wall
• [somehow lower priority for org_archive stuff](#smhwlwrprrtyfrrgrchvstff TIDDLYLINK) [[org]]
• [2020-04-13] Show HN: Built an extension to put an end to paywalls and popups | Hacker News
• [2021-01-19] markdown bug #promnesia [[promnesia]]
• [2020-05-20] shell script - Launching a terminal emulator without knowing which ones are installed - Unix & Linux Stack Exchange [[promnesia]] [[emacs]]
• [2020-11-02] Repl.it - Node.js Online Compiler and IDE - Fast, Powerful, Free [[js]] [[datetime]] [[promnesia]]
• [2021-01-18] promnesia/TROUBLESHOOTING.org at master · karlicoss/promnesia
• [2021-01-03] html - CSS set default scroll position - Stack Overflow [[promnesia]]
• [2021-01-17] #annotation-talks
• [[fetch all new posts and match them against promnesia?]] [[promnesia]] [[hackernews]]
• [[I'd say the biggest problem with something like this is that it's a silo. You're… | Hacker News]] [[worldbrain]] [[interop]] [[promnesia]]
• [2020-05-02] Vision, Mission & Values — 2020 Update - WorldBrain.io - Medium [[worldbrain]]
  • [2020-05-09] hmm, this storex hub thing could be used to feed in promnesia data
• [[Potential social features (other people's annotations)]]
• [[demonstrate on some provider how it works as plaintext (e.g. grep), then as json and then as specific provider (and what do we get from that?)]]
• [[demonstrate indexing as json and as more specific instapaper?]] [[promnesia]]
• [[in contexts if you add extra text it may potentially mess with highlighting. will be resolved with fuzzy I guess]] [[promnesia]]
• [[It could be a tool similar to browser. Maybe you don't use it very often, but when you want it it's pretty good to have it, like browser history. Also it's a an actually working prototype unlike most of other snippet's I've seen]]
• [[not sure how to decouple in the future?]]
• [[motivaiton: If you're digging for some topic you find yourself increasingly running at the same people]]
• [[Crucial ability is jumping right into context (tweet/tg message/etc). That complements search really well]] [[infra]] [[promnesia]]
• [[Tweet from @jethroksy]] [[promnesia]]
  • [2020-04-27] ok, I think it ties it via ID or something.. so not so agnostic..
• [@dalmo3: Built this chrome extension to get your @hypothes_is annotations into @RoamResearch https://t.co/X3A1AxfFkg](#dlmbltthschrmxtnsntgtyrhysnnttnsntrmrsrchstcxxffkg TIDDLYLINK) [[promnesia]]
• [2020-05-24] Vivaldi browser v1.8 released, with calendar-style browsing history | Hacker News [[promnesia]]
• [[People clearly like curating bookmarks. I could pitch it]] [[toblog]]
• [andy matuschak https://twitter.com/andy_matuschak/status/1206055315000528898](#ndymtschkstwttrcmndymtschkstts TIDDLYLINK)
• [2020-04-28] tweet at mek?
• [[I tried to make sure it's as resilient as possible]] [[promnesia]] [[errors]]
• [[highlight css styling among features]] [[css]]
• [[attempt to implement uniform engine for annotation: not necessary for hypothesis to do so]] [[promnesia]]
• [[design: few things became apparent: realtime updates are tedious to implement, however it's way easier to make stuff recompute everything from scratch. kcache was motivated by that]] [[toblog]] [[promnesia]]
• [[design: There is no way to implement this without keeping the data locally (or to be more specific, in one place)]]
• [2019-08-25] demo: HN Classics https://posobin.com/hn_classics/
  • [2020-04-27] eh, red dots don't look nice here?
• [[I was to make an emphasis: it's not a vaporware!]]
• [[in the process I had to fix the python url extraction library…]] [[promnesia]] [[toblog]]
  • [2020-04-29] could add this to yak shaving section
• [[Ideally I'd want some sort of AI assistant that would detect relevant bits and highlight them]]
• [[benefit of using browser history (sqlite) databases: easier to sync them around (or if you can't be bothered to do google takeout every now and then)]] [[promnesia]]
• [[bug: database might be lock while creating index? probably because of periodic indexer]]
• [[exclude fonts]] [[promnesia]]
• [[was in some sort of infinite/super slow binding loop in the sidebar on discord, turned it off for now]] [[promnesia]]
• [[further directions]] [[promnesia]] [[toblog]]
• [[hmm, interferes? e.g. try highlighting over a 'visited' mark]] [[promnesia]] [[hypothesis]]
• [[need to log less in serve?]] [[promnesia]]
• [querying for datetimes accounting for tz SELECT norm_url, dt, src, (coalesce(datetime(substr(dt,1,instr(dt,' ')-1)), datetime(dt))) FROM visits WHERE rowid IN (SELECT rowid FROM visits ORDER BY RANDOM() LIMIT 1000)](#qryngfrdttmsccntngfrtzslclctrwdfrmvstsrdrbyrndmlmt TIDDLYLINK) [[promnesia]] [[sqlite]]
• [2021-02-07] Insidious Big Brother Database User Manual: 1. BBDB [[promnesia]]
• [2021-02-07] Latest Nyxt topics - Atlas [[promnesia]]
• [[right, url extraction: https://github.com/lipoja/URLExtract/issues/13#issuecomment-467635302 and also I guess cut off trailing dots?]]
• [2021-03-28] installable mobile addon promnesia – Add-ons for Firefox (en-GB) [[promnesia]]
• [2021-03-26] Convert to ES6 compatible library by AStoker · Pull Request 65 · apvarun/toastify-js [[promnesia]]
• [2021-05-03] agora: promnesia-howto [[promnesia]]

Blog post: https://beepb00p.xyz/promnesia.html

Project: https://github.com/karlicoss/promnesia

* data sources

social networks

hangouts (from google takeout)

Local browser database

Google takeout [[takeout]]

investigation what data is actually contributing to takeout

visited http://slatestarcodex.com/superintelligence-faq/ through google search
from there jumped to http://slatestarcodex.com/2018/03/07/open-thread-96-75/
err weird, both are displayed in google activity (online); but only first one is in Search/MyActivity.html takeout. But last on is in Chrome/MyActivity.html. Right.. so I guess Chrome/MyActivity is a bit more useful?…
ugh. whatever, just use them all, there is no downside.

youtube watch history from takeout [[youtube]]

here is something for youtube https://github.com/pixelventures/youtube-history-querier/blob/f6da9ac88f93f776d41fcc159fc5632f64210d06/python-parser/main.py

use likes! [[twitter]] [[hpi]]

not sure what to do about dates.. support None for datetime??

Slide database – visited posts? Same for hn [[reddit]] [[hpi]]

[2020-04-04] right, SEEN database got some stuff /data/data/me.ccrama.redditslide/SEEN

use my.pinboard [[hpi]] [[pinboard]]

org context – need to strip away todo state changes… [[org]]

[2020-01-25] http://www.lord-enki.net/links.html import enki's wiki? [[exobrain]]

android resquetime database? [[hpi]] [[rescuetime]]

do I need to process org archive? [[org]]

android search history? not sure if worth trouble… [[takeout]]

also wonder, maybe I should turn google search history off?

[2020-09-05] hmm, firefox on android has 'combined' view?? [[firefox]]

right, it seems to group stuff

select date, url, title, history_guid from combined where history_id != -1 ORDER BY history_guid

merge all of takeout histories? maybe as long it appears in both [[takeout]] [[hpi]]

wow, there is more stuff in takeouts [[takeout]]

shit ok, takeouts are a decent way to preload it…

2232156  2019-09-13 00:32   Takeout/YouTube/history/search-history.html
2365963  2019-09-13 00:32   Takeout/YouTube/playlists/likes.json

hmm, youtube 'watch-history' is ticking. but Youtube/MyActivity is fully backed up??? [[youtube]] [[takeout]]

[2020-05-01] plus.maths.org |

hypothesis extractor could generate a nicer title of the document?

google takeout – parse json instead?? [[hpi]] [[takeout]]

think how should archives (zip/gz/etc) be handled…

Guess time based on git blame

multiple twitter accounts? [[twitter]]

deliveroo orders

stackexchange? reuse votes and favorites

stackexchange gdpr has lots of visit data

[2020-12-04] although it's over the whole domain, so not super useful

right, I am not using chrome anymore, which means takeout history is less and less useful for me [[promnesia]] [[takeout]]

use twidump as extra source?! [[promnesia]]

figuring out history from window titles? [[promnesia]] [[arbtt]]

bit too excessive I guess… although nice correlating events to jump in history?

extract 'regular' links from markdown [[promnesia]]

i.e. ones without http prefix

* ideas

automatic interface [[hpi]] [[promnesia]]

• datetime
• url
• exclude Json (not sure how?)
• first go through some data sources and see how well it's gonna work

start with 'static' version

• datetime: figure out by annotations
  TODO shit gonna be tricky with tz aware/unaware stuff
• url: figure out by 'url'/'href' attribute name (types don't seem to work?..)

then switch

• datetimes by isinstance
• url: dynamically by checking if 'http' or something in strings? but might be tricky

would be interesting to unify [[webarchive]] [[promnesia]]

• https://transhumanist-party.org/2019/06/09/transhumanism-teleology/
• https://turingchurch.net/how-transhumanism-changed-my-views-on-teleology-231d24d7a3bd was deleted, but I have some instapaper annotations

* extension issues/improvements

clicking on # on a github page causes reloading highlights…

i guess just need to ignore explicitly for now

[2020-11-19] eh. on the one hand, what else it should do? fragment could be anything

and also doesn't happen anywhere else

improving highlights [[annotation]]

on this page, the whole body is highlighted because of 'How does it work?'
https://github.com/karlicoss/promnesia#how-does-it-work

extension: ok, highlight by source color in 'show visited' definitely makes sense! [[promnesia]]

performance: pagination should be configurable, i.e. if you run locally you might not need it at all

extract canonical from page url as well? not sure how to make it more friendly and responsive… [[cannon]]

[2019-08-31] document.querySelector("link[rel='canonical']").href;

[2019-08-31] Determining canonical url

sort of a standard is using link rel canonical tag.

[2019-09-03] eh, it's a bit tricky because requires extra content script execution and merging. And in 99.99% cases it's gonna be same as canonified url?

https://physicstravelguide.com/advanced_tools/gauge_symmetry#tab__faq this for example gives gauge_symmetry page as canonical

[2020-05-01] maybe use it first? only then fallback on default algo. could make it configurable?

trying to debug unresponsiveness after idling [[promnesia]]

• chrome.runtime.onSuspend.addListener – aaaah, ok seems that it's triggering (errors are logged to 'extensions' settings page, oddly)

  function handleSuspend() {
    console.error("Suspending event page");
    // handle cleanup
  }
  
  chrome.runtime.onSuspend.addListener(handleSuspend);
  

• persistent: true – let's see
  TODO wonder if it would also remove the need for bckg injector?

  ok, almost def has to do with it!

[2020-11-18] All | Search powered by Algolia [[promnesia]]

breaks this header..

[2020-05-25] tabs.onActivated - Mozilla | MDN [[promnesia]]

might need to respond to this

try loading stuff earlier, while URL updates [[promnesia]]

I guess if notification off, it's not really a problem. just make sure it's caching and not querying too much
and test it too?

bug: in chrome [[promnesia]]

open any site
open sidebar
press search
close search
icon goes to grey, showing 'no visits'

bug: icon disappears on ctrl-t/close , but manual tab swtich with mouse is fine?? [[promnesia]]

[2020-05-01] doesn't work with readability? Quantum Diaries [[promnesia]]

doesn't work with readability extension?
add an explicit "refresh" button?

bug: chrome: callbacks stop triggering after I reload the extension manually. ugh. Looks like due to the same callback hacking in background.js [[promnesia]]

[2019-09-09] right, I guess that's because first page that's opened on chrome is options_page…

maybe frontend pages need to 'ask' backend to register callbacks?…

need to defensify everything [[toblog]] [[webext]]

Expose db from the server?? [[malleable]]

[2020-05-11] [Alexander Flores on Twitter: "@karlicoss having a pane like that but scroll synced to content in article would be ideal for me" / Twitter](https://twitter.com/ajflores1604/status/1259752083315703810 pull)

having a pane like that but scroll synced to content in article would be ideal for me

usability: perhaps easier with overlay in the middle of screen? then there are no issues with displaying…

usability: expand all/collapse all

usability: display summary of available contexts on top?

display some sort of 'collapsed' dynamic summary with tags and timestamps only? maybe on the side, and you 'zoom' into it as you scroll

bug: on 'cancel' after blacklist, still shows notification that it added [[promnesia]]

ui: split in tabs: annotations + plain visits? maybe even split by provider?

ui: Sidebar

[2019-08-01] eh, ended up implementing mine for now…

[2019-07-07] jeremychurch/FixedContent.js: A mobile-friendly, jQuery plugin that persists sidebar content, nav, etc.

https://github.com/jeremychurch/FixedContent.js
hmm,that looks interesting. sticky

[2019-07-07] arkon/ng-sidebar: Angular sidebar component.

https://github.com/arkon/ng-sidebar
another sidebar, seems ok

[2019-04-19] ui: Designing a Personal Knowledgebase – A Curious Mix [[promnesia]] [[org]]

http://www.acuriousmix.com/2014/09/03/designing-a-personal-knowledgebase
eh, need to improve rendering of context. look at this link, for instance

[2019-12-26] i guess it was referring to unhighlighted links etc?

bug: show visited mess with grasp?? (try selecting over these marks and capturing text) [[promnesia]]

excludelist: www.site.com vs site.com [[promnesia]]

* github issues

extension: would be cool to have an option to just 'find' empty space on screen to overlay… seems hard though [[ml]] [[think]]

[2019-07-20] more generally, non-invasive? and not sure if ml is necessary for that at all…

[2020-09-13] Cannabidiol - PsychonautWiki

extension works weird on this page w.r.t. scrolling

multiple processes

use multiple threads for indexing…

[2020-11-08] ugh fuck, def useful for indexing code..

load more if page is scrolled? gonna be tricky…

* indexer: bugs/issues

hmm. if we switch db mode to 'update', then schema changes would break promnesia… [[promnesia]]

hmm, what if people used it with older orgparse? (suggest to update in release notes)

personal style settings

.promnesia *[data-sources*=code] .context {
  font-family: monospace;
}

/* tweak 'visited' marks: specify hex color here */
:root {
  --promnesia-src-sourcename-color: #ff00ff;
  /* e.g.
  --promnesia-src-reddit-color: #ff0000;
  or
  */
  --promnesia-src-twitter-color: #00acee77;
  --promnesia-src-notes-color: #00ff0077;
}

[2020-11-22] extension: twitter is pretty slow & logs are spammy Home / Twitter

fuck, seems that twitter updates all the time? when scrolling
also bindSidebarData (in logs) is quite slow and also logs are spammy

[2020-11-23] ugh, so also impacts 'show this thread'

[2020-06-07] inkandswitch/ksp-browser: Connect the things you already know in your browser. [[pkm]]

[2020-09-11] hmm, need to outreach them?

different notions of 'visited' [[think]]

one is actually physically clicking
anothe is visual, i.e. scrape off the links that were on the screen and store (kinda like web archiver/memex)

figure out what could possibly be the most interesting source of links for people? I'd imagine whatsapp?

[2019-07-15] reddit definitely. Slightly harder to motivate backups [[reddit]]

[2019-07-15] could integrate with my reddit backup script then? reindexing would trigger backup; seems kinda ok

[2019-08-31] pocket perhaps?

[2019-08-31] actually, pinboard; keep one backup in repository to test the extractor or demonstrate json extractor? [[pinboard]]

[2019-12-15] Andy Matuschak on Twitter: "@LiquidTextCorp I'm very excited to try!" [[motivation]]

<https://twitter.com/andy_matuschak/status/1206055315000528898 pull>

@andy_matuschak:  @LiquidTextCorp @spiralstairs
I'm very excited to try!
I worry about the opacity of the app container model. Conceptually speaking, I want the LiquidText canvas at the level of the OS! Across not just some PDFs, but also web pages, mail messages, etc. The model pushes towards little app silos—it's a bummer.

should tweet at Andy Matuschak perhaps?

[2020-03-22] It would be also great if it would work backwards aswell, eg.: "the link I found on twitter last week"

https://news.ycombinator.com/threads?id=karlicoss#22657236

I love this idea and I would def use it! It would be also great if it would work backwards aswell, eg.: "the link I found on twitter last week"

[2020-05-28] I mean it will once I integrate with the database properly…

[2020-05-03] [(7) Spencer Chang on Twitter: "@jborichevskiy @Twitter @Wikipedia @hypothes_is @RoamResearch Love this have been thinking in the same space where the content you consume is linked in-place to the content you (or the people you think are important) curate. Looking forward to seeing where this goes!" / Twitter](https://twitter.com/spencerc99/status/1256747294482825216 pull) [[ui]]

Love this have been thinking in the same space where the content you consume is linked in-place to the content you (or the people you think are important) curate. Looking forward to seeing where this goes!

[2020-05-06] [WorldBrain.io on Twitter: "We are starting the development of the Memex Reader, an offline-first Pocket-style reader for desktop and mobile + mobile annotations. https://t.co/5lUpNSImFe Is anyone interested in collaborating/contributing to this? We need some more (wo)man power. ❤️" / Twitter](https://twitter.com/worldbrain/status/1258031458356277249 pull) [[worldbrain]]

We are starting the development of the Memex Reader, an offline-first Pocket-style reader for desktop and mobile + mobile annotations.
https://notion.so/worldbrain/Reader-Web-Archiver-Mobile-Annotations-extension-mobile-4ce4576dd1154f3f87f33eb6830ecebf

Is anyone interested in collaborating/contributing to this?
We need some more (wo)man power.

[2020-06-22] [WorldBrain.io on Twitter: "We're working on sharing/collaboration features of Memex and would love to have your input! Watch a quick rundown on some early mockups: https://t.co/brncMElHFr We'd love to have you for a 30-min call to get a grip on your use cases: https://t.co/AT52JajznO" / Twitter](https://twitter.com/worldbrain/status/1275042929845903361 pull) [[worldbrain]] [[social]]

We're working on sharing/collaboration features of Memex and would love to have your input!

Watch a quick rundown on some early mockups:
https://loom.com/share/5d9173ccf63a4865a99c84481abd5347

We'd love to have you for a 30-min call to get a grip on your use cases:

[2020-06-17] [TrailHub on Twitter: "https://t.co/do5RBGjjk4 Ready to give StorexHub a whirl! Take @hypothes_is annotations, and bookmark the pages in memex so that they will show up as a liked page in memex and be indexed Add Page notes to Memex that link to hypothesis annotations https://t.co/SUk55b3ADn https://t.co/QsybMSwHxo" / Twitter](https://twitter.com/TrailHub1/status/1273226379757322243 pull)

Ready to give StorexHub a whirl!
Take @hypothes_is
 annotations, and bookmark the pages in memex so that they will show up as a liked page in memex and be indexed
Add Page notes to Memex that link to hypothesis annotations

[2020-03-21] westoncb/mymex

https://github.com/westoncb/mymex

Mymex[0] is an application designed for quick retreival of information from a variety of (web/local) data sources. One of its main goals is to make pieces of information located on the web easier and more reliable to return to. It does this by:
    automatically storing local renderings of web resources
    allowing tags and notes to be attached to web resources

demo: on malleable systems collective, with screenshot from the chat itself? [[promnesia]] [[social]]

usecase: reading hackernews 'new'

• interesting domains are highlighted
• users on which profiles I clicked before are highlighted (so I'm likely to pay attention to their posts)
• posts I've seen before are highlighted (so I get to boost them hopefully to discuss later)

demo: 'mark visited' on the hackernews top?

[2020-02-11] mek.fyi | Home

https://mek.fyi/#about-me

Historia (Provenance) #chrome-extension

fuck me! Some very similar goals

[2020-11-19] tweet at Mek after releasing new version?

demo: post again on /r/orgmode? with demos of highlights/translusion/backreferences?

[2020-11-19] demo: good demo for mark visited Subscriptions - YouTube

this would be a good demo for 'mark visited'

[2020-11-17] usecase: New Links | Hacker News

I think I've figured out how to read hackernews 'new' page

ugh, something's not working with release build in firefox??

Error: The storage API will not work with a temporary addon ID. Please add an explicit addon ID to your manifest. For more information see https://bugzil.la/1323228.

discuss on malleable systems how hard was it to modify youtube's website and inject annotations etc [[promnesia]] [[outbox]]

could also discuss with Oliver from worldbrain memex? [[worldbrain]]

demo: tweet HN new reading when I encounter some user I know

publish: release on lobsters? seems ok

[2020-11-21] control logging in content scripts

Installed it and it seemed to work! But it was spammy in the console and I'm doing webdev, so I had to uninstall it. Would reinstall if you could silence the debug stuff

ugh fuck

[2019-07-08] testing: unitest/History at master · scriptmasters/unitest [[hpi]] [[jdoe]]

https://github.com/scriptmasters/unitest/blob/master/%7E/.e2e-chrome-profile/Default/History

[2019-11-22] highlight: https://news.ycombinator.com/item?id=21403294

if 'most' of page matches, then highlight is prob. wrong?

[2019-12-27] I guess if I show it on HN, better get it right

test: for the end2end test, run it against a database indexed with an older (PIP) version

[2020-11-19] actually not super important; in most cases database is overwritten

docs: make sure it's possible to run merely by creading config, then index, then serve

test it on a fresh docker

Please visit https://editorjs.io/ to view all documentation articles. Base concepts

for demo, could just anonymize people? that would solve 99% of it [[promnesia]] [[demo]]

[2020-05-11] [deen-chan on Twitter: "@ajflores1604 @karlicoss @jborichevskiy I've implemented this (augmented sidebar) and it's quite hard to come up with something that works well for all sites. You quickly run into sites which don't reflow, assuming a certain layout. But that was years ago. Responsive design is a lot more common these days so might work" / Twitter](https://twitter.com/sir_deenicus/status/1259792881679818752 pull)

deen-chan
@sir_deenicus
Level 5:
Replying to
@ajflores1604
@karlicoss
 and
@jborichevskiy
I've implemented this (augmented sidebar) and it's quite hard to come up with something that works well for all sites. You quickly run into sites which don't reflow, assuming a certain layout. But that was years ago. Responsive design is a lot more common these days so might work

[2020-07-05] Guide to using filemagic — filemagic 1.6 documentation

Before installing filemagic, the libmagic library will need to be availabile. To test this is the check for the presence of the file command and/or the libmagic man page.

[2020-05-20] I remember using this software last time, it is wayyyy~ too buggy, it stalls, cr… | Hacker News [[promnesia]] [[worldbrain]]

I remember using this software last time, it is wayyyy~ too buggy, it stalls, crashes, and slows down the browser. Also that import feature is actually crawling the site, beware if you are using a proxy or something with rate limit.

fucking hell. figure out how to fix version exposed during the adhoc install….

[2019-12-12] demo: Digital Tools I Wish Existed :: Up and to the Right — Jonathan Borichevskiy

https://jborichevskiy.com/posts/digital-tools
potentially good example of highlights
also they don't look quite well in dark mode?

[2020-03-28] maybe make a screenshot out of it?

[2020-05-20] Introduce Yourself - Thinking Tools [[social]] [[publish]] [[promnesia]]

[2020-04-29] integrate in Memex? [[worldbrain]]

post automatic demo gifs [[publish]]

doc: minimal example to import something in promnesia, with crontab examples etc [[promnesia]]

maybe it should be emacs:// ? not sure [[mimemacs]] [[promnesia]]

[2020-05-21] or editor:// ??

[2020-05-29] ok, so it seems that emacs:/// works and is backwards compatible, which is kinda good news

for editor, could set it up as mime type? dunno.

[2020-05-30] right. so I guess I know what to do

• During the indexing, detect either. If detected emacs:, then warn about deprecation and suggest to remove
  Need to think how to accumulate indexing warnings and show in the end… but later I guess
  If only old script detected, use emacs://, otherwise use editor://
• still can't use line=, col= because extension wouldn't support it. Or maybe it's fine actually!
  Version it!

[2020-05-29] could add to doctor?

[2020-11-01] maybe mimes should be configured & tested in the frontend? could add to the extension troubleshooting page

[2019-12-08] motivation:

https://news.ycombinator.com/threads?id=grblovrflowerrr&next=17598881

j2kun on June 1, 2018 [-]
How is this like the demo? This looks like "unify all N productivity apps; now I have N+1 productivity apps!"

demo: of 'mark visited' [[toblog]] [[promnesia]]

also use some guessing magic to quickly disable it? could test on https://pinboard.in/popular/
or just hide stuff that linked more than once on the page?

[2020-05-16] hmm, make sure visited marker/css is tweakable? I suppose it needs to be in the 'main', sidebar section. confusing…

if I make marker larger instead and the same color as links, it might be easier to filter out visually?

proromnesia-visited::after {
    content: "⚫XXXXX";
    color: #FF4500;
    vertical-align: super;
    font-size: smaller;
    user-select: none;
    position: absolute;
    z-index: 100;

actually even this works quite well

.promnesia-visited {
    background-color:
    red;
}

TODO filter: invert, but don't think it's possible without js?
border actually works quite well!
this isn't supper pretty, but enough to visually glance and notice

[2020-05-16] post about it – why don't we abuse a human brain which is much better at pattern recognition than computers (so far)

publish packed zips in releases; assemble automatically on ci [[ci]] [[webext]] [[promnesia]]

remove 'duration' from the database? hopefully no one uses it yet…

[2020-05-14] actually I might need it later.. if I populate them from #arbtt

[2020-01-13] usecase: l3kn/org-fc: Spaced Repetition System for Emacs org-mode

https://github.com/l3kn/org-fc

drill

very nice, apparently ran into that guy on merveilles and was able to track it in search!

[2019-11-01] ui: Shtetl-Optimized » 2016 » April

https://www.scottaaronson.com/blog/?m=201604
maybe, make the notification more subtle?

[2019-11-04] allow to be configured via CSS

[2019-11-04] how to automate it actually?

[2019-11-09] bug: Introduction - Everything I know

https://wiki.nikitavoloboev.xyz/?q=gree pull
constantly reloads on this page :(

[2019-11-12] eh. fair enough, it's changing URL. Not sure if there is much that can be done?

[2019-12-25] except if cannon works clientside. Ugh!

[2019-12-07] can be solved temporarily with proper blacklisting

be less annoying about errors.. not sure how though [[errors]]

[2019-08-31] just implement different options for notifications?

[2019-09-01] quick option do disable/enable notifications; later find some generic component to snooze them?

[2019-09-09] yeah, I guess they'd be ok on sidebar? Basically, make sure sidebar always slides since it's convenient for settings, access to search etc.

Def need way to make it less spammy on network errors on phone. System wide notification is too much [[errors]]

test going to different url in same tab; not sure how it would work…

support stuff without tz? or at least warn and maybe instert as utc. or just pass forward local tz that's ok for most people

[2019-08-29] mozilla/webextension-polyfill: A lightweight polyfill library for Promise-based WebExtension APIs in Chrome [[webext]]

https://github.com/mozilla/webextension-polyfill
Use this thing?

[2019-09-10] handle when pages aren't available better

https://tuxspace.net/@Qwxlea
check when pages are not available…

[2019-09-23] need to implement at least popup

[2019-09-05] lukeed/tinydate: A tiny (349B) reusable date formatter. Extremely fast! [[datetime]] [[js]]

https://github.com/lukeed/tinydate

[2019-07-14] Surprisingly Turing-Complete - Gwern.net

https://www.gwern.net/Turing-complete#on-seeing-through-and-unseeing
anchors are good examples of 'direct' visits and siblings?

I guess sort of killer feature would be 'hierarchical' visits (e.g. facebook pages, reddit comments, github. etc. for reddit will require some manual rules against canonified urls)

[2019-07-22] search kind of contributes towards that

stuff in database seems to be unnormalised (e.g. case). need some sort of checker … [[cannon]] [[togithub]]

[2019-06-16] hmm, do that I guess sqlite3 visits.sqlite "select norm_url,tag from visits where norm_url LIKE '%usg%'"

useful to populate from old backups (e.g pinboard) in case I delete articles/bookmarks [[motivation]] [[promnesia]]

• State "STRT" from "TODO" [2019-03-12]

filter links

chrome:// (history/apps/newtab
  newtab is interesting data though!

warn about anomally long histories?

would be nice to process links/pieces of information that I merely seen!

like hypothesis, but process the page server side and highlight against everything in the knowledge base [[hypothesis]] [[pkm]] [[promnesia]]

would be nice to know how I got onto the page… [[promnesia]] [[motivation]]

[2019-06-02] use chrome from_visit?

[2021-01-20] search basically does this

need some thing which normalises and merges messages? or maybe just disregard duplicate links from old and new backups… [[backup]] [[telegram]] [[promnesia]]

could analyse 'how did I get there' from my web archives [[archivebox]]

[2019-07-31] I guess I meant from plaintext search

publish: /r/dataisbeautiful post my visited urls stats?

Tweet from 𝔊𝔴𝔢𝔯𝔫 (@gwern), at Mar 10, 23:59 [[promnesia]] [[motivation]]

Several times in the past few weeks I or an acquaintance read something awesome only to realize we'd read it years ago & simply forgot! Another use for 'anti-spaced repetition' (https://t.co/jD4SsY6VBW): track great stuff & remind you to re-read it only 𝘢𝘧𝘵𝘦𝘳 it's forgotten.

<https://twitter.com/gwern/status/1104879445368864773 pull>

[2019-04-15] Pinboard: bookmarks for tswaterman tagged 'math'

https://pinboard.in/u:tswaterman/t:math/
I definitely need to integrate promnesia crawling with webpages backups, that way I'd have way more context

locators need short name along the full path

hmm, def looks like chrome is syncing bits of local database as well. ugh

polar got OK design for sidebar [[polar]] [[promnesia]] [[inspiration]]

custom search engine? [[search]]

'came from' is just a context. Would cover 99% of cases

tags are pretty useful…

Works really well in conjunction with axol (both dots and contexts) [[axol]] [[promnesia]] [[motivation]]

Example a complete guide for tagging

showvisited: show dots on selection? useful so you don't need to send too many queries. or anchors like surfingkeys?

display error in sidebar? [[errors]] [[ui]]

[2019-07-14] could be useful if we can extract URL; but encountered some issues while extracting context etc; could emit both visit and error

not sure, maybe I should just use native async/await? Most mobile and desktop browsers support it

[2019-07-15] generally figure out how much can I get away with using modern JS as opposed to webpacked

also def worth it for debugging and developing

go through existing urls and try to normalise them? [[webarchive]]

ught https://github.com/brookhong/Surfingkeys/blob/57fccbbeeb60ee2be0d2d60cfc50bd3aca3b0436/background.js#L1094 [[webext]]

switching frames works fine on https://web.hypothes.is/blog/annotation-is-now-a-web-standard/
though

[2019-07-07] shit. ok, so chrome experiment confirms that it just doesn't work for extensions.

[2019-07-07] shit. do I need bookmarklet or what???

[2019-07-08] Search · filename:places.sqlite https://github.com/search?p=2&q=filename%3Aplaces.sqlite&type=Code

[2019-07-14] mitchellkrogza/Ultimate.Hosts.Blacklist: The Ultimate Unified Hosts file for protecting your network, computer, smartphones and Wi-Fi devices against millions of bad web sites. Protect your children and family from gaining access to bad web sites and protect your devices and pc from being infected with Malware or Ransomware.

https://github.com/mitchellkrogza/Ultimate.Hosts.Blacklist

merge results together? e.g. search for inmotionmagazine

[2019-03-08] motivation: Nikita Lisitsa on Twitter: "@_bravit У меня так на stackoverflow / math.stackexchange бывает. Ищу ответ на вопрос, нахожу, читаю, ставлю лайк - "вы не можете поставить лайк своему собственному посту"." / Twitter [[promnesia]]

<https://twitter.com/lisyarus/status/1104104035588755457 pull>

[2019-07-08] Does Firefox ship with a new default for "history expires after…"? - Super User

https://superuser.com/questions/1114637/does-firefox-ship-with-a-new-default-for-history-expires-after

places.history.expiration.max_pages is maximum number of pages that are retained before pages are expired.
I had system disk fil

[2019-07-08] Does Firefox ship with a new default for "history expires after…"? - Super User

https://superuser.com/questions/1114637/does-firefox-ship-with-a-new-default-for-history-expires-after

Workaround
Consider using the extension Expire history by days.
Ironically this extension was written by the developer who changed the previous behaviour. See the blog post below.

for mimemacs, could probably support vim primarily (since all users will have it), it could even fallback [[mimemacs]]

[2019-07-23] demo: Best Mangal Bar & Kitchen delivery from Farringdon - Order with Deliveroo

https://deliveroo.co.uk/menu/london/farringdon/best-mangal-farringdon
allow breaking down important and unimportant schema parts from extensions?

rss blogs, easy to see what clicked and what hasn't [[rss]]

should show visual indication on 'no results'

when I merge together browser activity etc, I need to assert on some historic entries to make sure timestamps match properly

would be too annoying to deal with transitions I guess, considering that it's not needed too often. Just rely on jumping into timeline?

[2019-08-01] ??

open in new tab from search?

how did I get here – perhaps shouldn't mix context and visits??

yeah, searches useful to have in db for context; even though they'd almost never match and normalise

yeah, show url split in parts; then when I click one of the parts in queries more data?

direct link to css from the extension?

[2020-04-28] I guess this is possible judging by greasemonkey

could expand the grouped things on grey area press? [[promnesia]] [[togithub]]

maybe use two locators? one for linking and one for debugging

[2019-12-26] eh?

moz-extension://b966cab9-aded-44e8-b116-a900ab825442/search.html?timestamp=1561844122.151

looks sort of messy. should collapse and somehow make more unique…

hmmm author_url could be useful? [[hpi]] [[reddit]]

media_embed only got html
secure_media_ contains more stuff and could actually be useful

ugh. some have both media and secure_media??

bug: on android doesn't respond well on filter clicks, and overall looks a bit weird..

shows notification on navigation within page when clicking anchors http://super-memory.com/articles/sleep.htm#biphasic_sleep ,perhaps shouldn't do that…

enable back options to always show dots on visited urls?

[2019-08-31] uBlock/manifest.json at 6c34b3c3c96756b6db7ff2f3a0394472d81cde3e · gorhill/uBlock [[webext]]

https://github.com/gorhill/uBlock/blob/6c34b3c3c96756b6db7ff2f3a0394472d81cde3e/platform/webext/manifest.json

"optional_permissions": [
   "file:///*"
 ],

allow per-device settings? [[webext]]

bug: on restoring tab, shows notification twice. weird

in debugger looks like code just starts running at for (const action of (await actions()))

[2019-09-22] doesn't seem to do with hacky injector backed initialization either. odd

[2019-09-22] also only happens in firefox, apparently…

https://slatestarcodex.com/2014/03/17/what-universal-human-experiences-are-you-missing-without-realizing-it/

looks like some visits duplicate; and also they aren't grouped

[2019-07-30] hmm it triggers on clicking anchors. not sure it's a good idea? also add to end2end tests

https://beepb00p.xyz/annotating.html#org000001b

local sync vs cloud? Not sure how to handle this properly; maybe have different profiles?

[2019-09-09] test: fabianonline/telegram_backup: Java app to download all your telegram data.

https://github.com/fabianonline/telegram_backup

fabianonline/telegram_backup

clicking on title causes page reloading, also shows notification etc. a bit spammy. I guess for now having setting to prevent context notification is OK

fails on firefox.com domain; fair enough, but perhaps need to blacklist it?

missing host permission – very consistently reproduces on test_visits if you close/reopen tab

shit, tag map didn't work with context visits

have 'primary' settings profile and just think about rest later [[webext]]

on android, maybe should only request on sidebard click? doesn't display icon anyway… then basically it'd solve systemwide notification thing [[promnesia]] [[togithub]]

:invalid selector and input validation [[blog]] [[webext]]

[2019-10-19] Re: [fregante/webext-options-sync] Race condition in OptionsSync constructor

Highlight blacklisted and paywalled websites

Like ft, technology review

[2020-11-19] with mark visited it's much easier now.. could even have a special source and apply special style to them?

[2019-11-04] performance: vinta/awesome-python: A curated list of awesome Python frameworks, libraries, software and resources

https://github.com/vinta/awesome-python

Loggi

pretty slow on thins page… I guess highlights

[2020-11-14] hmm, now it's fine but it highlights a bit too much info? not sure what to do about the fragment normalisation…

[2019-11-04] promnesia/hypothesis.py at master · karlicoss/promnesia

https://github.com/karlicoss/promnesia/blob/master/src/promnesia/indexers/hypothesis.py

# TODO what I really need is my hypothesis provider... is it possible to share somehow?
for x in annotations:

I think I need to figure out how to make them standalone basically? Need some sort of defensive policy for dependencies?

[2019-11-09] On todo lists | beepb00p

http://127.0.0.1:8000/pkm-todos.html
bindSidebarData fails on quick refresh

Tweet from Will Manidis (@WillManidis), at Nov 26, 15:44

I spend my life copy/pasting links and screenshots of interesting content to friends. Solving infrastructure level information-transfer problems like this is also core to accelerating human progress.
@shohinigupta built something amazing to solve this

<https://twitter.com/WillManidis/status/1199337719295381509 pull>

[2019-12-26] ugh, tweet is gone??

ui, phone: don't really have to tweak body? as it's easy to hide/show sidebar [[promnesia]] [[togithub]]

I don't like some twitter account I'm following: they are whining too much or whatever [[twitter]] [[promnesia]] [[motivation]]

but why did I follow them in the first place, there must have been good reason?

Build client only version as demonstration?

query database

sqlite3 promnesia.sqlite 'SELECT norm_url, group_concat(src) FROM (SELECT norm_url, src FROM visits WHERE context != "" ORDER BY norm_url, src) GROUP BY norm_url'  | grep instapaper | grep notes | less

org-mode: comments from LOGBOOK might be useful.. [[org]]

for search might be good to display original link??

e.g. how did I get here?? https://mortoray.com/2019/06/11/a-failed-experiment-with-python-type-annotations/

[2019-12-15] bug: Теория вычислимости — Викиконспекты

http://neerc.ifmo.ru/wiki/index.php?title=%D0%A2%D0%B5%D0%BE%D1%80%D0%B8%D1%8F_%D0%B2%D1%8B%D1%87%D0%B8%D1%81%D0%BB%D0%B8%D0%BC%D0%BE%D1%81%D1%82%D0%B8
not sure, shouldn't show popup on every transition… have a timer or something?

Could map different people onto the same 'entity'?

after loading browser with previously open tabs and then closing them; shows 'host permission' warning

should allow running against live database? really why not, could be nice for exploring

Maybe git tracked server config is not so bad

Use hub/click? And maybe docker to isolate [[docker]] [[python]]

Would be rad if it was possible to simply run it against GitHub repo straight from ui

[2020-01-25] Robot&AIWorld on Twitter: "Here's more footage of MIT's Mini Cheetahs cavorting, frolicking, back-flipping, playing soccer and generally acting fun and cute, courtesy of the Biomimetic Robotics Lab @MITMechE https://t.co/8ZQzDvCDVW" / Twitter

<https://twitter.com/RobotAndAIWorld/status/1192429991813881856 pull>
should be handled, I liked/RTd it on twitter

Allow raw settings backup [[webext]]

[2021-01-20] this should really be easier…

shit, google docs trigger a lot of reloads.. really should debounce

duplicating search page – fucking hell. still happens on firefox…

chronic versioning? [[project]]

ugh. subdomain needs to be banned..

www.services.online-banking.hsbc.co.uk

[2019-12-27] Adventures in WhatsApp DB — extracting messages from backups (with code examples) [[promnesia]] [[whatsapp]]

https://medium.com/@1522933668924/extracting-whatsapp-messages-from-backups-with-code-examples-49186de94ab4
wow, someone managed to get whatsapp export working?

[2019-12-30] What am I meditating for? In Pursuit of A Definition of Meditation - Mark Koester

http://www.markwk.com/what-is-meditation.html
Highlight domain differently?

warning about module size…

https://webpack.js.org/guides/code-splitting/

not sure about tabs vs activeTab permission [[webext]]

e.g. if the page is updated in the background (e.g. youtube video), do we want to refresh promnesia stats? although it's basically only limited to youtube

[2020-02-15] maybe instead I could simply check last requested url on tab switch?

ok, I guess I could make "tabs" an optional permission; by default only request stuff via click on a popup (that requires 'activeTab'?)

shit, request on every tab switch is not good…

[2019-08-10] right, chrome doesn't support android extensions. could use bookmarklet or something?? https://stackoverflow.com/a/10606887/706389

as a docker app?

go through code and create issues?

eh, web indexer doesn't really work for relative links…

combine with eye tracking, then could know how much time was spent reading something…

would be nice to get stuff collected by axol to promnesia database [[axol]] [[promnesia]]

shit, 2mb per single page (10K visits) sucks…. wonder if should use some different protocol? Or gzip defeats the purpose?

[2019-12-05] Erik Torenberg (@eriktorenberg) / Twitter [[motivation]] [[demo]]

https://twitter.com/eriktorenberg
ran into twitter account recommendation (by michael nielsen, clicked it, found out I've already got few liked tweets by that guy

use dev webdriver??

profile = webdriver.FirefoxProfile('/home/xxx/.mozilla/firefox/abadadw.dev-edition-default')

demonstrate on that place with great bread [[promnesia]] [[demo]]

[2020-01-25] hyperhype/hyperscript: Create HyperText with JavaScript.

https://github.com/hyperhype/hyperscript

[2020-03-30] Troubleshoot extensions, themes and hardware acceleration issues to solve common Firefox problems | Firefox Help [[webext]]

https://support.mozilla.org/en-US/kb/troubleshoot-extensions-themes-to-fix-problems#w_checking-extension-settings

maybe add sidebar menu on mobile only? anyway, need to add blacklist

'mark visited' works really nice on the phone

doesn't work in private windows (I assume because of the getActiveTab thing)

common usecase – I follow someone, then I'm annoyed by their tweets and try to figure out why had I followed them in the first place?

Maybe we just need a proxy that logs literally all of http requests.. That way history could be agnostic

shit, webdriver only took screenshot of half of the frame….

not compatible with gh: style links…

[2020-04-07] usecase: Idle Words https://idlewords.com/

could test on twint?? [[twitter]]

usability: keyboard navigation [[webext]] [[malleable]]

https://github.com/karlicoss/promnesia/issues/13

[2019-08-01] ugh. surfingkeys is capable of working on my iframe, but can't switch

look somewhere along
https://github.com/brookhong/Surfingkeys/blob/57fccbbeeb60ee2be0d2d60cfc50bd3aca3b0436/background.js#L1091
https://github.com/brookhong/Surfingkeys/blob/f27a6cd30df285b05d78922a201b9e71912edd0e/content_scripts/front.js#L362

[2019-08-26] at least write about it in readme/faq

could draw a small 'visits/contexts' tree? would be fun..

maybe backend could update rules now and then.. [[cannon]]

Breakdown of contexts by parts of virtual url in the sidebar

attemt to set up autorealoding [[promnesia]] [[webext]]

web-ext run --verbose in the extension folder

had some issues
changes were detected on touch, but not on build
clean plugin should be handling properly, i.e. keeping dist/ dir and only removing files
https://github.com/johnagan/clean-webpack-plugin/issues/106
tried commenting/uncommenting clean plugin, and then it stopped happening????

in the future, try to investigate it by going to dist/ dir and trying ls in terminal

Write a post demonstating which bits should not be part of extension [[malleable]] [[promnesia]] [[toblog]]

Codejar/codemirror – let the user choose the editor
Linkify – third party link hihlighter
What else???

Hmm. Sending fragment can be unsafe?

I think this part is a little off:
> Perhaps you have no sympathy for web applications that store sensitive data in query strings, as that’s widely recognized as an insecure pattern. The URL fragment is more serious. That otherwise is a safe way to store sensitive information, so it’s alarming to see a third-party library sending a copy to an external server.
> Firefox Send and Mega.nz are both examples of popular web apps that use the URL fragment to store client-side encryption keys so that users can save end-to-end encrypted files to the cloud without the server ever having access to the underlying data.
The URL fragment is not designed to be any more secure than anything else in the URL, it's just a funny quirk of how web browsers evolved that it doesn't happen to be sent to the webserver. That popular platforms are (mis)using it to pass information without that information hitting their webservers is unfortunate. But it doesn't mean that the URL Fragment is somehow special or should be thought of as "secure" - that's not a guarantee that the URL scheme makes.
For example, those fragments will easily appear in browser history for anyone else who uses your same device...

use arbtt to match the browsing history? that would be fun… [[arbtt]] [[hpi]]

readme: probably, better to install it via pipx? so HPI etc are easier to upgrade? [[python]] [[promnesia]]

[2019-11-21] raxod502/mercury: Emacs interface to Facebook Messenger [[facebook]] [[hpi]]

https://github.com/raxod502/mercury

[2019-12-02] could be useful…

[2019-09-22] 1397667 - "No matching message handler" error when tabs.update().then(tabs.executeScript()) [[webext]]

https://bugzilla.mozilla.org/show_bug.cgi?id=1397667

[2019-09-22] 1290016 - tabs.executeScript in webRequest.onCompleted gives "Unchecked lastError value: Error: No matching message handler" [[webext]]

https://bugzilla.mozilla.org/show_bug.cgi?id=1290016

visual web scraper? geoffrey litt mentioned generalised nocode stuff (Pavel's question)

[2020-05-07] eslint-plugin-no-unsafe-innerhtml - npm [[webext]]

be more informative; show full history or at least last visit and potentially sources (e.g. hypothesis)

maybe icons for mobile/desktop?

attempt at agnostic url extraction (sha ae889fa0fb99f683cd1ba6192a3b55d11a481558) [[reddit]] [[promnesia]] [[hpi]]

I mean not sure what I feel about it. it's very adhoc in the first place, and still requires some hardcoded knowledge about useful and useless fields. I suppose not worth it at this stage

demonstrate jumping to the message on android? ironically it works better on phone [[promnesia]] [[demo]]

time spent? only for chrome apparently [[promnesia]] [[demo]]

could make a screenshot from a test_chromevisits test

Annotating post [[worldbrain]]

I have been using Memex for more than a year now. Here are the things that really annoy me
- occasional freezing and sudden disappearance of your bookmarks
- no real way to programmatically access your Memex database. I know they have released the storage backend, but the lack of helpful documentation is a deal-breaker.
- lack of collaborative annotation (the way Hypothesis does)
- only few results in search results!

[2020-05-20] Memex/manifest.json at develop · WorldBrain/Memex [[project]]

"omnibox": {
    "keyword": "m"
},

nice idea…

[2020-05-23] Programming Inside a Container | Hacker News [[docker]] [[hpi]] [[promnesia]]

what if I could apply :visited state for links that I marked as seen?

settings – add visual cues for dots/ visits/contexts/sources

[2020-05-13] Native Messaging - Google Chrome [[webext]] [[promnesia]]

[2020-05-27] Show HN: Obsidian – A knowledge base that works on local Markdown files | Hacker News

[2021-01-20] some people apparently already used against obsidian db?

Lesson [[promnesia]] [[publish]] [[think]]

Maybe a big lesson is that I need to write about ideas, frustrations and experiments earlier.
I could have released promnesia on hn a year ago and by now could repost again! Crazy!

Hmm, maybe do not highlight stuff from archived org files/deleted instapaper/etc?

Hmmm. I guess src map is going to be js hook for dynamically transforming visits? Provide a small dsl

Introduction to Marionette — Firefox Source Tree Docs 78.0a1 documentation [[webext]] [[promnesia]]

https://firefox-source-docs.mozilla.org/testing/marionette/Intro.html pull

tried disabling logic for single background page – doesn't help even with persistent background..

[2019-05-24] useful to have links just added to instapaper to know that you are planning to read them there

[2020-04-29] TypeError: can't access dead object - JavaScript | MDN

shit.. wonder if it could happen because of messing with window. or something??

[2020-05-05] type alias to union is invalid in runtime context · Issue #5354 · python/mypy

usecase: find all occurences of the bug in your code (e.g. if it got fixed)

docs: mention that I'll be posting about breaking changes in releases

[2020-05-29] even better to do this in commits and simply reference stuff in releases? or, sync releases with a file in repository

[2020-06-03] Using Service Workers - Web APIs | MDN [[webext]]

[2020-06-03] highlights: Quora Answers by David Pearce (2015 - 2020) : transhumanism with a human face

not great here

[2020-11-14] hmm, performance OK, but it throws ruandom erros now??

They record where you come from, what pages you visit, how long you stay on each, where you click and where you go next.

imagine if this information was available to you instead
Against an Increasingly User-Hostile Web - Neustadt.fr
in context

[2019-07-28] Firefox doesn't recognize telegramdesktop's tg link. /r/firefox [[telegram]]

I bumped on this post today while I was trying to find the solution for the org capture protocol. It's been a while since you posted it, but no one posted the solution, so I guess this might be helpful.
The thing is that any new protocol must be introduced by clicking on the link instead of passing the address to the location bar directly. Once you connect the new protocol with the app it will work regularly.
* Go to `about:config`
* Add new boolean named `network.protocol-handler.expose.tg` and set it to `false`
* Create the link by opening an empty tab and typing the following in the location bar
`data:text/html,<a href="tg://resolve?domain=Bold">Link</a>`
* Click on Link and Firefox should ask you to choose the program
* Check the box in order to remember it for future use

[2020-05-25] [Dmitry Bobrov on Twitter: "@karlicoss No idea about the service, but I do have experience with Chrome extensions if that’s what you mean" / Twitter](https://twitter.com/dvbobrov/status/1264995425175711745 pull)

No idea about the service, but I do have experience with Chrome extensions if that’s what you mean

[2020-06-11] doc: Share database between machine? · Issue #114 · karlicoss/promnesia

reuse readability extensions for better content highlight?

could use both for reddit cumulative state and promnesia browser exports [[hpi]] [[reddit]] [[cachew]]

Post archive org highlights [[outbox]]

merge mode – could combine public and private data

[2020-05-11] ESLint v7.0.0 Released | Hacker News [[promnesia]] [[webext]]

There is a rule in eslint that warns you when a promise is dangling and hasn't been handled.

PLEASE USE THAT RULE. So many bugs in the JS world is because of dangling promises.

[2020-05-25] [Korobochka on Twitter: "@karlicoss Via Twitter?) For the last ~6 months I am developing a Chrome extension at work, can take a look as well." / Twitter](https://twitter.com/korobochka191/status/1265029486782922752 pull)

Via Twitter?)
For the last ~6 months I am developing a Chrome extension at work, can take a look as well.

version detection – thing in cachew works pretty well? perhaps it might not work without installing, but otherwise should?

HPI/promnesia: a whitepaper? [[publish]]

ok, could use some design inspiration from ampie [[ampie]]

• floating sidebar. make the position configurable, so it doesn't overlap with other potential extensions
• hmm. for firefox it knows the time spent?? I wonder how
  uhoh. seems like it's keeping track via js… https://github.com/posobin/ampie/blob/da50f2c5abc1b59de2fc8379b13c7d6d813577e7/src/main/ampie/content_script/amplify.cljs#L328
• site icons – although not sure how to extract them.. from the browser history?
  def site icons for sources? Could specify sources favicons in js config, maybe
• moz-extension://d55e97e7-da63-4685-a3bc-dffc93092eb4/hello.html – very nice… perhaps could include promnesia tutorial
• list of all links on the page – nice!

[2020-09-25] (20) Anyone use Promnesia? | Building a Second Brain

hm, for serving reuse config, but allow to split config bits to make defensive?? not sure..

could use pandoc for serving HTML?

one downside is that need to return both? one for display, another for matching highlights?

about realtime indexing (give a demo of entr??) [[totweet]] [[promnesia]]

support child visits etc for inbrowser history

dynamic JS hook only in dev mode?

how to display warnings in the very end? extract from warnings module maybe?

using generators to defer UI updates? [[toblog]] [[promnesia]] [[js]]

ideally configuration would be kinda like a grid? for every (almost) source and for every function

sharing links with surfingkeys? e.g. they could both filter crap links

sidebar isn't opening if new tab, then google search (firefox)

check blacklist right before querying backend as the last resort?

for excludes, 'do not mark this link' and 'do not mark this element'… the latter is gonna be way harder [[togithub]] [[promnesia]]

usecase: when I follow someone on github/twitter/etc I'll make a quick note. Next time I'll know why [[demo]]

[2020-11-10] inkandswitch/ksp-browser: Connect the things you already know in your browser.

Backlinks

A backlink is a link to this page from somewhere else you've been. It might be a blog post you read, or another page on the same site. It could also be a link saved in a local note on your computer.

[2020-11-19] Essays · Gwern.net

tweet at gwern & share

[2020-11-18] Watch later - YouTube

share youtube watchlist with highlights

[2020-11-17] Merveilles it's pretty dynamic and might not work well with 'mark visited'

some websites

basically log them along with the URL

[2020-11-17] Withings body+ vs. Withings body cardio : QuantifiedSelf

for mark visited blacklist, need to normalise? not sure…
I guess ideally, yes..

[2020-11-14] What forces layout/reflow. The comprehensive list.

[2020-11-16] TweetDeck

sidebar isn't closing on this page..
unless you press 'close'?? wtf..

hmm, could serve the exlcudelist from the backend???

[2020-11-22] The File System Access API: simplifying access to local files [[promnesia]]

visited marks might impact ellipsized content

https://github.com/karlicoss/promnesia

faq: some pages may prevent extensions on working on them

Content Security Policy: The page's settings blocked the loading of a resource at inline ("style-src").

[2020-11-22] eh, not sure if this is the case with extensions? bookmarklets (like Hypothesis on HN – yeah)

Add donation link, to bastien?

if I use sidebar, it will work even on websites that forbid html injections? [[togithub]] [[firefox]]

maybe I need something a bit more discrete for contexts notification…

shows host error exceptions in firefox on pdfs [[promnesia]] [[togithub]]

https://www.fordfoundation.org/media/2976/roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure.pdf

[2019-12-26] could be disabled via blacklisting *.pdf probably?

process archive as well? [[promnesia]] [[org]] [[togithub]]

could hide annotations via tags? [[promnesia]] [[togithub]]

backlist: tampermonkey?

[2020-11-16] eh?

[2020-05-25] iorate/uBlacklist: Blocks specific sites from appearing in Google search results [[webext]] [[promnesia]]

good support for blacklist, reportedly good code quality

add some throttling for the notification at least

[2020-02-25] added suppressing..

[2020-03-23] maybe, it's better just to turn it off by default…

prefill with domain name? [[promnesia]] [[search]]

ublock content picker is awesome… would be nice to find one..

blacklisted: allow to force loading (from sidebar)

sidebar doesn't work while search is open :( :(

[2020-05-02] Plain text linkifiers comparison (work in progress) (http://jsbench.github.io/#54315d74264c857856c73f18c81278dc)

closing tab before it's finished loading causing error. I guess that's fine and shouldn't be defensified..

not sure how to handle desktop notifications vs builtin? I'd prefer to use system as far as we can I guess but they are not easy to fine tune [[webext]] [[togithub]]

perhaps I could sketch some simple library/api similarly to how android does that?

host permission on closing and reopening tab?

right, switching to highlight.js/codejar does make it 110kb (vs 170 kb)

#hypothesis adds special element <hypothesis-highlight>? wtf??

[2020-03-02] windows 10 - Disable Chrome to ask for confirmation to open external application everytime - Super User [[promnesia]] [[mimemacs]]

https://superuser.com/questions/1481851/disable-chrome-to-ask-for-confirmation-to-open-external-application-everytime

Disable Chrome to ask for confirmation to open external application everytime

ignored entries should be colored as black [[promnesia]] [[togithub]]

could merge (e.g. chrome and takeout) both on client and server; not sure which would be best

replace fixture with tmp_path like in orger?

[2019-09-15] bug: raw.githubusercontent.com/karlicoss/dotfiles/master/surfingkeys/config.txt

https://raw.githubusercontent.com/karlicoss/dotfiles/master/surfingkeys/config.txt
shows error on this file very consistently..

[2019-09-22] ugh. works fine in dev firefox and doesn't even have sources on release?

[2019-09-23] also seems to be fine in chrome, however doesn't display kkj

process confidences and somehow add this information in visit line

show with discarded url get params? [[promnesia]] [[togithub]]

[2019-06-02] eh??

[2019-12-27] I guess I meant ignore normalisation etc

pinboard last seen

I guess needs a pinboard provider?

fuck :D google takeout had json formats for activity… [[backup]] [[promnesia]] [[takeout]]

use really really old takeout and see what happens

ctrl-r doesn't seem to update extension icon for stale tab

Motivation for browser history: its just easier to grab than google takeout data. I just need to merge them together i suppose [[motivation]]

option to disable dots on the page (permanently or temrporary)

Maybe desktop integration?

[2019-12-26] what?

[2021-01-20] like a remembrance agent maybe?

backend: URL extraction via NN?

hmm it sort of around when I'm clicking links on the page.. would be interesting to see how that works

[2019-08-05] eh?

Watch "There is No Algorithm for Truth - with Tom Scott" on YouTube [[promnesia]] [[motivation]]

https://youtu.be/leX541Dr2rU

Why did I add this video? I don't know the speaker or anything

[2019-11-21] eh, not sure..

people who followed you?

[2020-11-20] not sure what I meant here?

wonder if it would help me to figure out what was the video https://www.youtube.com/watch?v=Z-R2xO3phVM

title is missing; i suppose because I should have merged takeouts

fallback in sequence?

e.g.
jq -> dumb json
bs4 -> grep

right, spend a bit debugging sidebar with 10K visits

I mean, even if I don't add the container to the hierarchy immediately, rendering (in particular, createElement) takes a bit
I guess it's not very reasonable to expect 10K items to render really really fast

1. Limit number of items returned from the backend? think about it later
2. Perhaps it's indeed better to delay it until opening the sidebar?

tell about fake systemd? [[toblog]]

Would be fun to run the backend on Android..

need to fix compare db to allow for small relative changes? at least for pinboard and reddit

pydeps src/promnesia [[python]] [[hpi]] [[promnesia]]

Hmm wonder if could adapt promnesia for Next browser?

I agree :-)
The user should have the power. Also the interface should be as powerful as possible. One such browser that embraces this philosophy is Next https://github.com/atlas-engineer/next. Source: biased author

[2020-04-24] usecase: compute-space/compute-space.md at master · jauntywunderkind/compute-space

found that bookmark in my notes without any comments. why did I add it?
click on local – found rektide's post!

[2020-04-24] usecase: westoncb/mymex

useful for processing bookmarks
e.g. I've got a list of websites/projects I bookmkrked.
sometimes they are a bit experimental and cryptic, and I struggle to remember why I added it in the first place
here – click on last visit results in HN post

[2020-05-06] Firefox for Mobile now supports NoScript, PrivacyBadger, HTTPS Everywhere | Hacker News

I fear they're going to force it as the release version while supporting just 1% of most used extensions, forgetting that it's a fat-tailed distribution. I use video background fix to play youtube with a minimized browser (no I don't want to use a separate app if I don't have to).

At that point I'm just going to switch to Brave.

[2019-01-19] Switching over to communicating via private server

ok, keeping the items in memory was definitely a bad idea..
looks like the way to go is just running a server in parallel. the benefit is being able to run against the remote server, so not having to sync the links

• the database thing – better, but still kinda clunky to update etc
• also, we could optimize easier if it's not js, presumably some users might want to store a lot of links. Also more opportunity for interaction with FS and realtime watch
• also normalisation code doesn't have to be shared with JS

[2020-06-30] Native messaging is extremely broken and has bad documentation. I never got repl… | Hacker News [[webext]]

Native messaging is extremely broken and has bad documentation. I never got replies to my bug reports about it and gave up and just used websockets.

browser history: clicked 'forget about this site' for google.com by accident.. wonder if it's gonna erase stuff

[2020-07-06] Linux kernel coders propose inclusive terminology coding guidelines, note: 'Arguments about why people should not be offended do not scale' • The Register

Words to be avoided include "slave", with suggested substitutions such as secondary, subordinate, replica or follower, and "blacklist", for which the replacements could be blocklist or denylist.

[2020-11-06] jeanralphaviles/comment_parser: Python module to extract comments from source code files of various types.

hmm, try on this first? also wonder why its libmagic version is fixed??

[2020-11-08] target browsers insteadd of Node, remove old react stuff, update eslint babel · karlicoss/promnesia@c8380e0

test_blacklist_builtin

failed on github ci selenium.common.exceptions.NoAlertPresentException: Message:
test_{blacklistbuiltin}

[2020-11-16] uBlock/webext.js at 5aca41960373656c79d547b1a983f053e6adb633 · gorhill/uBlock

async chrome apis examples

bug: right, seems that extension disconnects after a timeout in chrome based browsers

e.g. leave a tab open for several minutes and do smth else – it stops responding

[2019-09-01] keepassxc-browser/manifest.json at develop · keepassxreboot/keepassxc-browser [[webext]]

https://github.com/keepassxreboot/keepassxc-browser/blob/develop/keepassxc-browser/manifest.json

"applications": {
   "gecko": {
       "id": "keepassxc-browser@keepassxc.org",
       "strict_min_version": "52.0"
   }

[2019-09-08] figure out why is this necessary?

hmm, need to overwrite database in order to handle schema updates? not sure what would be a clean way of doing it…

[2019-08-07] I guess similar to cachew?

[2019-09-03] ok, although haven't tested that behaviour

I really need page notes… e.g. reasons why I removed the bookmark and whether I already visited it..

maybe, set up inotify to update the database?

maybe show yellow icon if we only have chrome visits? so in a sense, green is more 'interesting' visits. also, maybe blink with green for a while (and have a timer whether we blinked in the past hour or something)

[2019-06-01] need automatic generation for differently colored icons…

[2019-06-02] distinguish contexts as well

[2019-08-08] just needs a better documenting..

share code with js …. ugh, all the projects I found really suck… [[ffi]] [[malleable]]

I guess just share the 'normalizing regex'?

highlight/color certain tags (configure that in settings)

[2019-06-01] suggest tags? maybe by querying from api, although presumably there wouldn't be too many. make the config YAML

make sure it searches in archive…

• State "DONE" from "TODO" [2019-02-16]

Use cachew for browser history [[promnesia]] [[cachew]]

[2020-05-27] sshkarupa/url-handlers [[mimemacs]]

[2019-09-01] vinta/awesome-python: A curated list of awesome Python frameworks, libraries, software and resources

https://github.com/vinta/awesome-python

Mistune - Fastest and full featured pure Python parsers of Markdown.

at first, make filters private when I release it

[2019-10-12] ok, they do save a bit of space for me, e.g. 575Mb vs 376 Mb. I guess keep my private ones for now and keep them empty in config, can resolve later

[2019-10-27] fuck I'm so glad I'm keeping notes! Would have never remembered why db size changed otherwise..

reddit link in context

would be also useful to highlight visited posts, e.g. here https://www.lesswrong.com/posts/vwqLfDfsHmiavFAGP/the-library-of-scott-alexandria [[promnesia]] [[demo]]

• State "DONE" from "TODO" [2019-02-16]

attempt to use prismjs

not sure what was wrong, but it didn't work with codejar. line numbers were broken on line breaks
kinda suspicious, he gives prism in demo, but uses highlight js himself..

const PJ = await import(
    'prismjs/prism.js'
);
const Prism = PJ.default;
await import(
    /* webpackChunkName: "codemirror.css" */
    // $FlowFixMe
    'prismjs/themes/prism.css'
);
TODO use highlightElement? like example 3 here https://medv.io/codejar

wonder if I can embed in a page, as an Iframe?

[2019-12-26] would be hard because of need to login, etc…

[2020-02-18] Upload Add-on :: Promnesia :: Add-ons for Firefox

[2020-10-28] thesephist/histools: A collection of tools for generating data visualizations from browser history data

release: suggest to update the backend

later, would be nice to propagate it to the extension ui…

need to figure out how to keep error traceback/context.. [[python]] [[promnesia]] [[errors]]

I failed miserably at reaching out to Worldbrain/Hypothesis to discuss whether we should collaborate :worldbrain:hypothesis

and instead engaged in a coding/refactoring streak. oh well

How to run as a proxy server? So it would work with any apps

Tweet from @posobin [[promnesia]] [[inspiration]]

<https://twitter.com/posobin/status/1330005221410529282 pull>

@posobin: I really like it when an interface responds to holding some key down, and gets back when you let the key go, the immediacy of that is cool. Added such a feature to ampie: hold alt/option and the tooltips for all the ampersand badges to open up, let it go and they disappear. https://t.co/tw7bVltXXX

[2020-11-07] ugh. looked in codemirror src, but unclear how to optimize the size further. it's just big shrug

[2020-05-06] agnostic extraction – extracting context is pretty much impossible [[promnesia]] [[hpi]]

[2020-11-01] profiling: so overall it seems that half time spent in parsing org-mode, half in url extraction. meh

time py-spy record -o profile.svg -r 1000 -- scripts/promnesia demo data/notes/

results in source-org-profile.svg

[2021-01-20] this py-spy thing is very nice [[performance]] [[python]]

https://www.freecodecamp.org/news/lossless-web-navigation-spatial-model-37f83438201d/ <https://merveilles.town/@aynish/105139837128207627 pull>

email mek

mark visited – maybe for ignored links add an attribute/explanation or something

what if I add 'fake' history into browser database? wonder if that would work…

[2020-11-26] Nyxt

Tree based history- lossless by design.

[2020-11-27] showvisited: Новые серии / MyShows.me

visited marks shift the left menu

[2020-12-14] Gephi - The Open Graph Viz Platform [[timeline]]

[2020-12-14] could draw en edge if the visits are close in time?

[2020-05-17] All Time - Hacker News Top Links

[2020-01-22] Playing around with Chrome's history https://gist.github.com/dropmeaword/9372cbeb29e8390521c2#chrome [[chrome]]

[2020-12-05] interop: DIYgod/RSSHub: 🍰 Everything is RSSible [[rss]] [[promnesia]] [[wildcard]]

hmm extractors for different sites could be reused

[2020-12-06] Org-roam User Manual

• The roam-file protocol:
• The roam-ref protocol:

[2020-12-06] Org-roam User Manual [[grasp]] [[promnesia]]

fucking hell.. protocols and mime handling are a mess

[2020-12-06] Newsletter 10 - Neovim v0.4.4 - Neovim

moves the top navigation here

[2020-12-13] showvisited: Mile End Climbing Wall

moves around links here

somehow lower priority for org_archive stuff [[org]]

[2020-04-13] Show HN: Built an extension to put an end to paywalls and popups | Hacker News

highlight paywalled links?
although out of scope, probably

[2021-01-19] markdown bug #promnesia [[promnesia]]

ok, i figure it out, in md , only the [](url) could be recognized a true data, nor plain text link or paragraph link

[2020-05-20] shell script - Launching a terminal emulator without knowing which ones are installed - Unix & Linux Stack Exchange [[promnesia]] [[emacs]]

else try xdg-terminal
else try x-terminal-emulator

[2020-11-02] Repl.it - Node.js Online Compiler and IDE - Fast, Powerful, Free [[js]] [[datetime]] [[promnesia]]

gh. tried dayjs.. but it doesn't seem to be able to handle timezones specified by offsets. ugh.

var dayjs = require('dayjs')
require('dayjs/locale/en')
var timezone = require('dayjs/plugin/timezone')
dayjs.extend(timezone)
var customParseFormat = require('dayjs/plugin/customParseFormat')
dayjs.extend(customParseFormat)
dayjs('21 Jun 2020 12:58:01 -0500', 'DD MMM YYYY HH:mm:ss ZZ', 'en').format()

[2021-01-18] promnesia/TROUBLESHOOTING.org at master · karlicoss/promnesia

readme: add the 'check backend'?

[2021-01-03] html - CSS set default scroll position - Stack Overflow [[promnesia]]

canonical: stackoverflow.com/users/21886/richiehindlesources : codestackoverflow::users/{ids}/favorites::[]::answers::[]::owner::linkstexport/data/stackexchange_20201204T020325Z.json04/12/2020, 02:03:25

ugh..

[2021-01-17] #annotation-talks

Weird.. but thanks for letting me know. There is a promnesia doctor command that troubleshoots some common problems, I can suggest users to run it with python3 -m promnesia doctor, so it can detect the missing .local/bin and warn them!

fetch all new posts and match them against promnesia? [[promnesia]] [[hackernews]]

even simple rss + local email could work

I'd say the biggest problem with something like this is that it's a silo. You're… | Hacker News [[worldbrain]] [[interop]] [[promnesia]]

https://news.ycombinator.com/item?id=21310988

• [2019-11-23] xx

  I'd say the biggest problem with something like this is that it's a silo. You're suddenly 100% reliant on them providing the right tools and functions to access your data in the way you want/need. And if your needs/preferences change, then you're entirely reliant on whether the silo has foreseen the new use case. I already have an existing knowledge base (that not only consists of webpages, but also org files, videos, pdf's, etc.) that's accessible and synchronized across multiple devices - and while having a complete searchable history of all my browsing would be fantastic, there's no way to integrate it into my system (or any other system) with my own tools.

[2020-05-02] Vision, Mission & Values — 2020 Update - WorldBrain.io - Medium [[worldbrain]]

[2020-05-09] hmm, this storex hub thing could be used to feed in promnesia data

I guess my big difference is the

• backend
• normalization
• context and 'child visits'

I've already had the UI. Next step would be seeing if we can cooperate. I'll try to reach some people in Memex and chat perhaps.

Potential social features (other people's annotations)

demonstrate on some provider how it works as plaintext (e.g. grep), then as json and then as specific provider (and what do we get from that?)

demonstrate indexing as json and as more specific instapaper? [[promnesia]]

in contexts if you add extra text it may potentially mess with highlighting. will be resolved with fuzzy I guess [[promnesia]]

It could be a tool similar to browser. Maybe you don't use it very often, but when you want it it's pretty good to have it, like browser history. Also it's a an actually working prototype unlike most of other snippet's I've seen

not sure how to decouple in the future?

motivaiton: If you're digging for some topic you find yourself increasingly running at the same people

Crucial ability is jumping right into context (tweet/tg message/etc). That complements search really well [[infra]] [[promnesia]]

Tweet from @jethroksy [[promnesia]]

<https://twitter.com/jethroksy/status/1233408971437821952 pull>

@jethroksy: I wrote a feature for Org-roam I really like! Have notes tied to arbitrary web pages, use a Firefox bookmarklet to instantaneously bring you to the correct notes file. Template flexible, can prefill with all kinds of things! https://t.co/KpMGazgS0I

[2020-04-27] ok, I think it ties it via ID or something.. so not so agnostic..

@dalmo3: Built this chrome extension to get your @hypothes_is annotations into @RoamResearch https://t.co/X3A1AxfFkg [[promnesia]]

<https://twitter.com/dalmo3/status/1229733827595001856 pull>

[2020-05-24] Vivaldi browser v1.8 released, with calendar-style browsing history | Hacker News [[promnesia]]

This. Susan Dumais at Microsoft had a project called "Stuff I've Seen" [1] in the early days of desktop search. In a nutshell, it's indexing and information retrieval on the data that you use in your daily life -- web browsing, calendar events, etc.

I've often thought about trying to re-implement some of their ideas with a local caching proxy or a browser extension. If there's an open source attempt at this already, I'd love to hear about it.

People clearly like curating bookmarks. I could pitch it [[toblog]]

andy matuschak <https://twitter.com/andy_matuschak/status/1206055315000528898 pull>

[2020-04-28] tweet at mek?

@mekarpeles hey, I've ran into your google doc https://docs.google.com/document/d/1QlrVpAl59gmJD7Kl--q3uyfZo936iZIb45fim6EV53k <button class="pull-url" value="https://docs.google.com/document/d/1QlrVpAl59gmJD7Kl--q3uyfZo936iZIb45fim6EV53k">pull</button> , I'm working on a browser extension that's addressing Problem 1 https://github.com/karlicoss/promnesia#readme

I tried to make sure it's as resilient as possible [[promnesia]] [[errors]]

in js, it's very easy to swallow errors. at best it just stops code halfway and renders system weird (exceptions would only be visible in extensions logs which no user would look at)
worst case you end up with inconsistent state (TODO reformulate) or break some invariants
I've wrapped everything in TODO defensify
But that manifested in lots of spurious errors for various edge cases, so I had to deal with it. tests; fuzzing

highlight css styling among features [[css]]

attempt to implement uniform engine for annotation: not necessary for hypothesis to do so [[promnesia]]

design: few things became apparent: realtime updates are tedious to implement, however it's way easier to make stuff recompute everything from scratch. kcache was motivated by that [[toblog]] [[promnesia]]

design: There is no way to implement this without keeping the data locally (or to be more specific, in one place)

[2019-08-25] demo: HN Classics https://posobin.com/hn_classics/

nice, 'show dots' works really well for that

[2020-04-27] eh, red dots don't look nice here?

I was to make an emphasis: it's not a vaporware!

It's not even a prototype! It's a but raw, but it works for me, and there is no reason it won't work for you apart from some infrastructural overhead.
I would like to raise a discussion on making this easier. I appreciate it may still be far from being possible to use by non-technical people, but at least my goal is so people who know/capable how to git clone and run from jobs could set this up

in the process I had to fix the python url extraction library… [[promnesia]] [[toblog]]

[2020-04-29] could add this to yak shaving section

Ideally I'd want some sort of AI assistant that would detect relevant bits and highlight them

for now we can at least do some sort of fuzzy highlighting?

benefit of using browser history (sqlite) databases: easier to sync them around (or if you can't be bothered to do google takeout every now and then) [[promnesia]]

Two sided?
The only downside is you won't have timestamps unless you write them out

bug: database might be lock while creating index? probably because of periodic indexer

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: CREATE INDEX index_norm_url ON visits (norm_url)]
(Background on this error at: http://sqlalche.me/e/13/e3q8)

exclude fonts [[promnesia]]

'While indexing exobrain/exobrain-logseq/static/fonts/IBMPlexMono-Text-Latin1.woff', 'No extractor for suffix .woff, mime font/woff

was in some sort of infinite/super slow binding loop in the sidebar on discord, turned it off for now [[promnesia]]

further directions [[promnesia]] [[toblog]]

I'd like to solve some general problems, so we can all benefit

• cannon
• timeline storage etc
• how to embed metadata into the page?
  usually
  empty space detection

hmm, interferes? e.g. try highlighting over a 'visited' mark [[promnesia]] [[hypothesis]]

need to log less in serve? [[promnesia]]

maybe allow changing logging level via a signal??

querying for datetimes accounting for tz SELECT norm_url, dt, src, (coalesce(datetime(substr(dt,1,instr(dt,' ')-1)), datetime(dt))) FROM visits WHERE rowid IN (SELECT rowid FROM visits ORDER BY RANDOM() LIMIT 1000) [[promnesia]] [[sqlite]]

[2021-02-07] Insidious Big Brother Database User Manual: 1. BBDB [[promnesia]]

someone from Matrix nyxt suggested it's similar?

[2021-02-07] Latest Nyxt topics - Atlas [[promnesia]]

right, url extraction: https://github.com/lipoja/URLExtract/issues/13#issuecomment-467635302 and also I guess cut off trailing dots?

[2021-03-28] installable mobile addon promnesia – Add-ons for Firefox (en-GB) [[promnesia]]

[2021-03-26] Convert to ES6 compatible library by AStoker · Pull Request 65 · apvarun/toastify-js [[promnesia]]

Convert to ES6 compatible library

[2021-05-03] agora: promnesia-howto [[promnesia]]