Hazel and DEVONthink 2 - Choosing a Target Database

In my previous post on using Hazel and DEVONthink together, I showed you a system I use to automatically get files and documents into the DEVONthink Pro Office Global Inbox.

Did you know, however, that is possible to import content directly into a database of your choosing? It DOESN’T have to just move to the Global Inbox?

Full credit to Alan Duncan of [ojisanseiuchi.com] for this nugget.

Here is how:

When you have configured your Hazel rules to identify the files that need to moved, add this AppleScript to the actions that will be carried out:

tell application id "com.devon-technologies.thinkpro2"
    set img_ib to get record at "/Inbox" in database "Business"
    import theFile to img_ib
end tell

This script will tell Hazel to take the content we have already identified and import it into my Business database. To change the target database, simply modify the name in the script.

Couldn’t really be any simpler at all could it?!

Thanks Alan.

Making DEVONthink and Hazel Play Nicely

Before I move on with this post, I'm going to make a couple of assumptions. I'm going to assume that you already understand that Hazel is an amazing application for macOS, that watches over folders in some form of virtual nanny-state, performing file actions on your behalf whilst you get that busy work done.

I'm also going to continue on the basis that you know about the awesome power of DEVONthink Pro Office, the Database/Paperless Office/Finder Replacement/All round swiss-army-knife-type-storage-app for macOS.

They are both amazing applications and I couldn't imagine having to live without either of them. However, after reading various forums and listening to some of my favourite podcasts, it seems that the general impression is that you can't have a workflow that includes them both. Using Hazel to rename files isn't possible within DEVONthink itself.

While the statement above is largely true, a small amount of lateral thinking can, however open up some possibilities.

I'm going to walk you through the setup I have employed to allow for the renaming of files with Hazel and subsequent import into DEVONthink Pro Office.

1) All files downloaded to my Downloads folder (the default)

2) Hazel watches this folder and automatically performs OCR on any PDF files that appear, using AppleScript and PDFPen Pro

3) These files are then renamed and moved to a Holding Folder I have in Dropbox.

4) Using the built-in Folder Actions for DEVONthink, a script runs on all contents of the folder. This script will Import, OCR and Delete the original file.

Dropbox Holding Folder

The first stage is to set up a Holding Folder. Now this Holding Folder has one purpose and one purpose only. It will perform a Folder Action that will Import a file into DEVONthink, perform OCR (Optical Character Recognition, making the contents of the document searchable) and Delete the original. This folder is the final place files will reside before import into DEVONthink.

The OCR that will be completed in this stage will be using the ABBYY Fine Reader Engine, arguably the most powerful OCR software on the Mac and built-in to DEVONthink Pro Office. We're going to be performing a 'lighter' (yet still powerful) OCR at an earlier stage, so as to rename the file effectively, however the final OCR will take place here.

Why have I specified that this should be setup as a Dropbox folder? Well, firstly, any folder connected to a Cloud Service is fine (Google Drive, OneDrive, iCloud) - I just happen to use Dropbox extensively. Secondly, I find myself scanning and submitting business expense receipts from iOS a LOT. I use PDFPen Scan+ to scan the receipt, I rename with a TextExpander snippet before sending it directly to my Dropbox Holding Folder. When it reaches my Macbook, the Folder Action kicks off and the file is imported into DEVONthink.

Assigning Folder Actions

Assigning the Folder Action to your newly created Holding Folder is a simple process.

• Open Spotlight
• Search for Folder Action Setup
• Click the + button on the left hand pane. Navigate to the Holding Folder you wish to assign the action to.
• You will be presented with a list of scripts to assign. Select Devonthink - Import, OCR and Delete.scpt

Please note, you must have the scripts for DEVONthink installed. If you don't see any DEVONthink references in the box of assignable scripts, install them by clicking on DEVONthink Pro Office in the menu bar, select Install Add-Ons and tick the check box for Folder Actions Scripts.

Great stuff. You now have a folder that will act as a direct conduit between yourself and DEVONthink that is accessible from anywhere you have an Internet connection.

Hazel Rules

OK. Now, we need to set up some Hazel rules. The question we need to ask ourselves, however, is where should these rules be applied? We've set up a Holding Folder, which is going to Import, OCR and Delete thanks to the Folder Action we assigned, yet this folder needs to be the last stop for the file - the final conduit between Finder and DEVONthink. Therefore I apply the majority of my Hazel rules to the default Downloads folder. These rules will carry out some OCR (using PDFPen Pro from Smile Software and subsequently rename the file based on the results.

I'll refer to you an earlier post on Date Matching with Hazel which runs through the process of scripting the OCR, looking for a date within the contents of the document and subsequent renaming.

The only part you need to add to the rule, in order to import into DEVONthink, is to move the file into your Holding Folder once renaming/tagging etc is complete. (Remember, DEVONthink recognises Finder tags, so you can use Hazel to pre-tag your files before they hit your DEVONthink Inbox)

You can make a lot of changes to the workflow above to suit your specific needs. I know that this works for me and apart from the odd tweak here and there, I'm very happy with how it runs.

There will be a video posted soon that walks through this process in finer detail - sometimes screencasts work better than text ever can, so watch this space and I'll let you know when it's ready!

Using Predefined Groups in TextExpander

When playing about with some of the features within TextExpander for MacOS last week, I happened across a small feature which, I have to say, I'm a little embarrassed that I hadn't noticed properly before now (and I've been using TextExpander for a long time!)

For those of you who don't know (and you should - you really should), TextExpander is a great utility from the team at Smile Software that allows you to type short snippets/abbreviations and these will expand into longer form text, frequently used phrases, symbols, images - pretty much anything that can be created from your trusty keyboard. When you get slightly more advanced, TextExpander can perform calculations based on date/time, run scripts - it's a really powerful tool that you need to have if you want to maximise the time you are at your machine for.

PreDefined Groups

These were hiding from me in very plain sight. Clicking the + button in the bottom left hand corner of the main TextExpander window presents you with the option of adding a PreDefined Group. These are groups of snippets that the developers have kindly put together that are linked to a particular theme.

1) Accented Words - You may have need to type a letter, or communicate with someone who uses accented language. Now, rather than hunt for the correct key, you can use one of the predefined snippets

2) Emoji - Need I say more? What better than being able to type a phrase such as :turd: to be express your dissatisfaction at a more appropriate level.

3) HTML & CSS Snippets - Not being a coder myself, I have it on good authority from those in the biz that this is a pretty comprehensive list of snippets.

4) Internet Productivity Snippets - These will run AppleScripts to shorten URLs using tools such as Bitly.

5) Symbol Snippets - I wish I'd known about these before, as it took me ages configuring snippets for ⌘, ⌥, ⬆ etc. Turns out they existed all along, I just installed the app in a hurry...!

6) AutoCorrect Snippets - This is the only Predefined Group that I find myself adding snippets to! A list of some of the most common mis-spelt words are in here. As a fast touch-typist, I find myself making the occasional SNAFU. If I see any common words that I'm making mistakes with, I'll add the correct spelling to this folder. Real time saver.

There are Predefined Groups for AutoCorrect in many different languages.

So - I'm sure the majority of you knew about this, however for those of you who didn't - well, you're welcome.

DEVONthink Part 7 - Database Maintenance

Today I want to cover the distinctly un-sexy task of maintaining your DEVONthink Pro Office databases. It's one of those things that makes you initially want to yawn, yet as soon as you run into difficulties, you will be so glad you took the time to safeguard against disaster.

Internal Backups

By default, DEVONthink Pro Office creates regular backups of the databases index files. These are then used in the rare occurrence of the system being corrupted. These internal backups are kept directly inside the database folder. You can define how often these backups are taken by opening Preferences and clicking the Backup section.

These backups are NOT backups of the entire database. For these, you would need to use Time Machine or a similar third party application. I use BackBlaze for my off-site storage and Time Machine to back up the databases to my local NAS device, as well as exporting via a script (I come to that next).

External Backups

You can export all files contained within your DEVONthink Pro Office databases to external storage quite easily. Simply highlight all groups within your database with cmd+A, then go to File -> Export -> Files and Folders. You will then be prompted for a location to export your content to by way of a standard Finder window.

You can also use an AppleScript to backup all files and folders within a database and schedule it to run with Automator if you want to take it up a notch. The built-in Export script is great and you get it by clicking Scripts -> Export -> Daily Backup Archive.

This will compress all files and folders into a nice little zip file ready for you to transport anywhere you like. I have a Hazel rule that runs which checks for the presence of this .zip file in my Holding Folder and creates copies on my NAS and OneDrive for Business folders. Yes, this may be overkill, but I'm not going to lost my data and that helps me sleep.

Verifying and Repairing

Databases get damaged over time. It's going to happen. The most common inconsistencies I receive are when I index an external folder - sometimes I will receive errors, saying that files are missing. These errors can be repaired (not always, but most of the time) by going to Tools -> Verify and Repair. DEVONthink will then check the consistency of the database and warn you of any errors, as well as letting you know the action that has been taken where appropriate.

Rebuild Database

Sometimes, the errors that appear above can't be repaired via the Verify and Repair tool. One of the options you have is to go to Tools -> Rebuild Database. Here, DEVONthink will export all of your files and group and re-import them into a new database. It sounds like a lengthy task but I've never had a database rebuild take longer than a minute and, invariably, it solves any problem I had.

Restore Backup

You can, of course, restore manually form one of the backups defined earlier. Go to Tools -> Restore Backup and choose from one of the point-in-time backup slots available. (Please note - if you elect to Rebuild Database, a backup will be created automatically. That is why I have 4 restore options available in the screenshot below.

Thanks for reading. My next post will be the last in the DEVONthink series and will look at the different Preferences settings that can be applied.