#27: Lookup Image Reverse Using

With all the fuss about AI in recent months (the latest being OpenAI teasing some futures with GPT-4o, and potentially raining on Google’s I/O parade that followed the next day), it’d be easy to overlook that elements of artificial intelligence have been infusing the software and services we all use every day, for years. Google are even revisiting an old Microsoft brand too

Text, handwriting and speech recognition, language translation, cognitive understanding – they’re all milestones to what people might think represents true AI, and using elements in conjunction with massive amounts of data has given us some incredibly useful capabilities.

One such is being able to do a reverse image search – the idea that if I have a thing, or a picture of it, how can I find out more about it, or where it’s being used elsewhere online? Copyright holders might want to search for unauthorised use of their materials, or we can even use the technique to tell us more about what our phone camera is looking at.

clip_image002Visual Search

The Bing search engine has had a visual search feature for many years – that’s right, some people do still use it, even by choice rather than because it’s the default or due to a nag screen.

The simplest way to use Visual Search (if you’re using Edge browser and Bing is your search default) is to right-click on an image and choose the Search the web for image option, which feeds the picture into the visual search page.

clip_image004

This will show you other places on the web that feature the exact same image (and in different sizes, too) as well as displaying other, similar images.

clip_image006

If the pic you started with is a recognizable place or person, it may offer a suggestion of what/who it is, with links to further info..

clip_image008

If you’re not using Edge and/or Bing isn’t your default, you can use it by copying the image you want to the clipboard (or grabbing a portion with the Snipping Tool), the go to the Bing homepage and click on the Image search icon.

clip_image010

Just paste your image in from the clipboard (press CTRL-V, or if you’ve switched on Clipboard history, WindowsKey-V will let you choose from previous ones too) to run it through image search. You can, if need be, adjust the area being searched for, by clicking the Visual Search icon towards the bottom of the main image, then dragging the handles to crop the area you want – picking out a single person in a group photo, for example.

clip_image012

Google offers the same kind of functionality, too – from Chrome with Google search as the default, choose Search image with Google,

clip_image014

… or try search by image from the homepage…

clip_image016

… and paste the selected image in there.

You’ll see slightly different results from the different search engines, so it’s definitely worth trying both out. The Bing user interface is arguably nicer than Google’s but in the end, it’s the results that count.

Mobile apps

When it comes to dealing with the real world rather than online photos, smartphones clearly provide a great starting point. The main Google app has the same initial image search UI as the web site but lets you point your phone camera at something and extract text from it, identify what it is and find out more. The Bing mobile app (and Microsoft launcher on Android, if you use that) does similar things but nowhere near as effectively, judging by the results it returns.

clip_image018

There are many specialist mobile apps for identifying specific things, like differentiating between a plant or a weed, but it’s worth trying the Google app first.

clip_image020

The Bing mobile app purports to do similar things, too…

clip_image022

Nul points

Coming back to looking for pictures, if you don’t get any meaningful results from search engines when trying to match an image, there are specialist services like TinEye, which offer deeper reverse image search.

Take this image from a blog post many years ago, before mobile video calls were really a thing*. Searching Bing/Google for it brings nothing of note, but TinEye found various sites which took part of the image and repurposed it – various nutjobs used the image in “news” that the next gen iPhone was going to have video conferencing capabilities, neatly overlooking the fact that the main subject of the photo had a curly-wired handset to his ear…

Screenshot 2024-05-16 160755

* in fact, Orange launched the SPV M5000 smartphone – aka HTC Universal – in 2005,

clip_image025

and it was the first 3G “phone” which had a front-facing camera for doing video calls. It wasn’t very good.

685 – Browser searching

Screenshot 2023-06-06 181351Research from a couple of years back showed that the most-searched-for term on Bing.com was “google”. While it seems crazy that people would type the name of a search engine into the search box of another, it’s possible they were entering “google” into a box on their homepage or even in the browser address bar, and that term was sent to bing.com as a query, rather than sending the browser to google.com.

If you’re using Edge and have Bing as the default search experience – other search engines are available – then you may see the prominent search box in your new tab page, but it’s worth remembering that the address bar at the top of the browser is also a search box. You can jump to the address bar in Edge or Chrome by pressing ALT+D, which also selects the current site’s URL (if there is one) so you can edit it or just replace by typing something else.

clip_image004If you start putting the name of a site into the address bar, you’ll be offered autocomplete suggestions from your favourites and your previous browsing history, so it may be very straightforward to jump to not just the website but a specific and previously accessed page within.

Entering a site name and pressing CTRL+ENTER will add the https://www. and .com bits so you don’t need to; therefore, to go to the BBC website, you could press ALT+D bbc CTRL+ENTER and you’d go there directly.

Although the address bar will ultimately use your default search engine to query a word or phrase that doesn’t appear to be a web site address, you can force it by starting to type ? in the address bar, then enter your search term after the question mark.

clip_image006Some sites will allow the browser to search within them by adding the site name and then pressing TAB. Whatever text you enter after the TAB will be sent to the specific search page of that site. Not all sites support this method, but many common ones do, like Twitter, Amazon, YouTube and more.

clip_image008Go to the search engine settings in Edge (or jump to the address bar and enter edge://settings/searchEngines) to see which sites are set up already. You can add your own “search engine”, which means you can direct Edge how to search within that site.

Click Add to include one of your own, using the appropriate site URL while replacing the bit where the search term is specified with %s – eg searching the OneDrive photos section for “dogs” would give a URL of https://photos.onedrive.com/search?q=dogs.

Give the Search Engine a shortcut name you want to use and then paste the modified URL and hit save. Now, in this example, typing photos | TAB | cats | ENTER would seach OneDrive for cat pictures.

If you are a Microsoft 365 user then you might be able – if it’s been enabled for your tenant – to search internal work documents and Sharepoint sites, just by typing work | TAB | etc. It’s on by default, but admins could also give you custom keywords / shortcut words too.

clip_image010Finally, on the topic of Searching in the browser, it’s possible to search across all the tabs you have open; start typing something in the address bar and you’ll see the option of filtering that search to apply to Work, history, favourites or tabs.

clip_image012Alternatively, press CTRL+SHIFT+A to kick the search off, type in the word of phrase you’re looking for and it will filter the list of current tabs to show only ones that match.

To quickly jump to that tab, use the up and down keys to select the one you want, and press Enter.

681 – AI Push Me Pull You

clip_image002As more AI hoo-hah continues to pour from the hype machine of big tech, some of the puffed-up services are starting to become more accessible. Even if it’s still badged as “preview”, Bing’s AI chat is now more widely available and further functionality will be along soon.

Stepping onto the bandwagon before it fully departs, AI technology behind Google’s Bard chatbot is being embedded into Search, as announced at Google’s I/O conference, whose theme was all about how everything is being re-engineered to embrace generative AI. El Reg has neatly summarized the keynote here if you’re interested to learn more (“We sat through the Chocolate Factory’s PR blitz so you don’t have to”).

Google announced copilot-like functionality for its cloud productivity suite while Microsoft unveiled the M365 Copilot preview that’s been running for a few months, is being extended further.

Not all is rosy in the garden of AI, however. Distinguished scientist Geoffrey Hinton, a Turing Award winner and widely-described as “Godfather of AI”, has walked out of his Googly role amid concerns that AI will become sentient and enslave or kill us all. Interestingly, Hinton did not sign the Elon Musk-backed petition to halt AI development, effectively saying that if those currently working on it were to stop, others would pick up the baton. Microsoft’s chief scientist agrees.

clip_image004Making AI pay for itself is one challenge that will need to be addressed, as the intensive computation required can be very expensive – costs of running ChatGPT are eye-watering, according to OpenAI’s boss, and reckoned by some to be in the region of $700K per day. Still, investors can’t get enough of it and OpenAI is piloting a $20/month ChatGPT Plus subscription.

The expanded M365 Copilot preview is a “paid-for” thing, and Microsoft’s Q3 earnings call did mention that Copilot will be additionally priced over and above whatever Office licenses a customer already has (though some AI related features will show up in E3/E5 licensed environments, such as the new Semantic Index which can be harnessed by Copilot but will be useful for giving more accurate search results even if Copilot is not in use).

Back in the present, there are some relatively new practical capabilities in both Bing AI and in the Edge browser’s discover feature, as discused in last week’s ToW. The Compose feature in the Sidebar lets you play with generating different types of written content, the kind of thing which will be integral to Copilot in all kinds of Office applications before long.

The Insights tab on the same Sidebar gives you more info on the page you’re currently looking at, from a summary of the key points of the page, to some background on where that site is accessed from, how likely it is to be reliable and more.

clip_image005

The core Bing AI search in a browser – in case you’re itchy about using Edge or its Bing Sidebar – has some new clip_image007capabilities, especially the integration of Bing Image Creator, which is available clip_image009separately from the AI chat function.

Another one of OpenAI’s groovy tools – Dall-E – generates images based on a text description, and Bing AI chat can feed directly into that.

The image generation capability is now multi-lingual (with over 100 languages supported). It will also soon be possible to upload images to Chat, so you could ask it questions about what’s in the image.

All free for now, but someday soon, we will need to pay the ferryman or the robot overlords will wreak their revenge.

680 – Edgy emails

clip_image001

The Edge browser has seen a lot of change in its life. Originally conceived as a successor to Internet Explorer, with its own modern web rendering engine and lots of additional features which are designed to complement the usage experience, like taking handwritten notes on top of a webpage or building a Reading List of pages or publications to come back to.

Later, the decision was taken to replace the browser with one based on a new core using Chromium, largely for reasons of compatibility and performance, but to carry on building new capability that would differentiate the new Edge browser from others that also use the Chromium rendering engine, including Google Chrome itself.

clip_image003Recent builds of Edge have a Sidebar which includes a load of apps and integrations – the goal being that it can help multitask on the web by sharing complementary information or functionality alongside the page the user is looking at; think a shopping widget that would compare prices of the product on the page you’re viewing, showing where else you could buy that same thing.

A recent update to the Sidebar has been the inclusion of the new Bing search, which adds some very cool relevance capabilities that would allow you to fire the current page content straight at the Bing’s AI engine to summarize, rewrite or explain the contents – selected text, or the entire page you’re visiting.

Here’s an example of a reasonably detailed blog article (from early 2021) summarized into a few key paragraphs:

clip_image004

One of the more useful integrations in the Sidebar is the Outlook app (individual icons on the Sidebar can be enabled and disabled through the settings option; you can also dock other sites which will appear in the sidebar, though not necessarily with the context of the page you’re currently looking at). A recent – and somewhat controversial – change means that when you click a link from an email in desktop Outlook on a PC, it will open in Edge and the Outlook sidebar will be shown alongside, displaying the email that you clicked it from.

Once you’ve got the hang of this feature, it’s actually pretty cool – especially if the email is offering some context about what you’re supposed to be doing on that page, or if it’s a densely-packed missive full of clickbait and other nonsense:

clip_image005

Why is it controversial? Well, the point is that the extra functionality is happening due to the Sidebar in Edge, so clicking a link in Outlook if you’re using a different default browser wouldn’t have the same effect. Outlook, therefore, has decided to send links to Edge even if that’s not your usual browser, to the chagrin of some netizens. Be careful with doing things that annoy some people.

If you’d prefer that Outlook and Windows respected your choice to send all your links to a specific, non-Edge, browser, then it’s fairly easy (if not exactly easily discoverable) to set that. Go to File | Options | Advanced within Outlook, and look for the Link Handling option, and change it to Default Browser. clip_image007This will mean opening the hyperlink in Chrome / Brave / Firefox / whatever, without the Sidebar doing its thing.

More change is on its way to Edge and Bing AI.

If you like Edge but would rather dispense with the Sidebar altogether, go to the “…” menu on the top right, select Settings | Sidebar and disable the Always show sidebar toggle.

You can use the same clip_image009settings UI to play with other behaviours in the various apps that are pinned to the sidebar, too.

To add or hide apps on the sidebar, just show it, right-click on something and choose Customize sidebar, or use the “Add or remove apps…” feature from the Settings | Sidebar screen.

clip_image011If you’d rather not to have the somewhat prominent Bing icon on the very top right of your Edge screen, look under the Discover section of this Settings UI, and if you flick the switch, the big blue b goes away.

668 – Ay Eye Ay Eye, Oh

clip_image002

Not Aye-Aye, Ally-Ally, nor Why-Eye, but Ay-Eye, as in A.I. And not the cheesy Spielberg flick. The tech news has been all about artificial intelligence recently, whether it’s ChatGPT writing essays or giving witty responses, to Microsoft committing another chunk of change to its developer, OpenAI.

Original backers of OpenAI include Tony Stark (who has since resigned from the board in order to discombobulate the world in other ways) and AWS, though Amazon has warned employees not to accidentally leak company secrets to ChatGPT and its CTO has been less than enthused.

clip_image004ChatGPT is just one application – a conversational chatbot – using the underlying language technology that is GPT-3, developed by the OpenAI organization and first released over 2 years ago. It parses language and using previously analyzed data sets, gives plausible-sounding responses.

Further evolutions could be tuned for particular clip_image006tasks, like generating code – as already available in PowerApps (using GPT-3 to help build formulae) or GitHub CodePilot (which uses other OpenAI technology that extends GPT-3). Maybe other variants could be used for interviews or auto-generating clickbait news articles and blog posts.

Another use for GPT-n has been unveiled in the New Bingan old dog has maybe learned some new tricks?

clip_image008You’ll need to join the waitlist initially but this could ultimately be a transformational search technology. Google responded quickly by announcing Bard, though Googling “Google Bard” will tell you how one simple mistake hit the share price. No technology leader lasts forever, unless things coalesce to there being only one.

Other AI models are available, such as OpenAI alternative, Cohere, and there are plenty of sites out there touting AI based services (even if they’re repainting an existing thing to have .ai at the end of it). For some mind-blowing inspiration including AI-generated, royalty-free music or stock photos, see this list.

642 – Finding work stuff

Data storage has become very cheap over the decades – a while ago, ComputerWorld wrote an article, saying that when it was founded 50 years previous, a 1MB hard disk would cost you $1M, and in 2017, that would work out at $0.02. 5 years later, $0.02 would get >1GB, more than 1,000 times as much.

clip_image002[4]This profusion has turned many of us into pilers – what’s the point of organizing data and deleting old stuff, be that files, emails, camera roll photos?

Outlook has a pretty good search function built in. OneDrive photos has some great organizing and filtering capabilities (like On This Day, or if you have GPS enabled on your camera/phone, you can easily group photos by the location taken from).

clip_image004[4]Still in OneDrive, there is also some AI-based tagging of your pics, which can sometimes be a bit hit & miss… but more often than not gets it about right.

While browsing “All Photos”, if you mouse-over to the right, you’ll get a scrollable timeline too (similar to the Windows Photos app), so you can quickly jump to a reference date.

Assuming you’re using Microsoft 365 / Office 365 at your workplace, there are other ways to find stuff that is more work-related, like documents, email and messages. One easily overlooked source is the “new tab” experience within the Edge browser.

clip_image006[4]The content on the default home page can be customized in a variety of ways, from choosing whether to show a background image or keep it clear; to displaying content from various “news” providers and clickbait advertisers that Microsoft News / MSN has elected to present to you, or hiding that altogether.

You can do some filtering of that content too, though for work purposes, many people may want to leave the page layout in “Focused”, which puts a link bar at the bottom and hides the content to be a scroll away.

Edge Profiling

If you have a “Work” profile (or you only have a single profile) and it is connected to your work account – ie your Microsoft 365/ Office 365 email address rather than your personal one – then you’ll see a “Microsoft 365” link within the list of content providers, which gives you a simple view of your most recent documents, SharePoint sites you visit and a whole lot more. To learn more about this Edge Enterprise tab, see here.

clip_image008[4]

Also, you’ll see the search box containing your company name – it uses Bing to search for whatever you put in there (somewhat controversially, regardless of what your default search engine in the browser is…). Edge doesn’t give you the option – like IE used to – of starting a tab with a completely blank page, though there are hacks to make that work.

clip_image010[4]If you stick with the standard new tab, it will also give you the choice of restricting your search to “Work”, so looking at documents and the likes. You’ll see a list of content sources clip_image012[4]displayed on the left side as tabs, allowing you to filter what you’re looking for or where you want to search.

There’s a fairly new one that searches “Messages”. At least for now, that means Outlook and / or Teams messages, but it could be really useful when trying to remember if a conversation you had was in email or in a Teams chat.

A quick way to jump to this section is to go to aka.ms/messages

– regardless of which browser you’re using, as it uses the Bing.com/work back end.

608 – Shopping Season

After clip_image002Americans celebrated “Turkey Day” yesterday, the phenomenon of Black Friday is now well underway. Originally coined for bricks & mortar retailers to kick off the shopping spree in the run-up to Christmas, it’s now exported around the world and applied across all retail channels with consumer electronics being routinely discounted for a few days (stretching through the weekend to what grew to be called “Cyber Monday”).

Around a year ago, the Edge browser debuted the Shopping feature which showcased vouchers from various sites you might visit – similar to the Honey add-in which offers coupons and vouchers proactively when you visit e-commerce sites.

clip_image004There have been some recent updates to Shopping, including a price tracking feature which tells you if a specific item has been reduced in price recently – as with all these things, YMMV depending on the retailer and your own location, but it’s certainly worth a look – find out more, and see which retailers are supporting the Shopping feature with coupons and price alerts.

There are other improvements on the way – including a rare UK-first rollout, of a Bing Shopping collaboration with Good On You to highlight ethically sourced fashion. Have fun on your shopping spree!

Tip o’ the Week 438 – Bing’s Visual search

clip_image002In an effort to attract users to the perennially under-rated search engine Bing, the service launched a “visual search” function some time ago (ToW 397). There have been some recent updates to mobile apps, enabling search capabilities from an image. Similar images will be identified or major subjects could be spotted, using AI technology to try to figure out what’s in the image, as well as find related info.

The Bing Search app and the Edge browser for both iOS and Android have been updated, as has the Microsoft Launcher for Android – each has added a little camera icon to the Search box, which makes it easy to take a pic and upload it to Bing for analysis. The Bing app can also scan QR codes and the Launcher can scan barcodes too. More features are promised for all 3 methods of app-based visual search, and for the Bing web site itself.

To use visual search in the Launcher, swipe from the left at the home screen and you’ll get the “Feed” page (a customisable summary of news, activities, apps, contacts etc), the top of which has a search bar with a camera and a mic (though if you’ve used the barcode scan in the Launcher, the camera will be replaced with a scanner symbol – just tap that as if to scan a new barcode, then tap the barcode with an X in the top right to revert back to camera).

If the results you get aren’t quite spot on at first, you could direct visual search to focus on a particular part of your snap – tap the magnifying glass in the top right, move the edges of the area to filter out any peripheral nonsense, and you may find results improve.

clip_image004

Tip o’ the Week 405 – Bye Bye, Bird’s Eye

clip_image001The Bird’s Eye view is/was one of the nicest bits in Bing Maps (get there quickly, at www.bingmaps.com – did you know you can type simply bingmaps in the address bar of Edge or Chrome, and press CTRL-ENTER to add the www and .com bits?).

clip_image003The Eagle-eyed amongst you may have spotted that some changes have happened… clicking on the drop-down control to change the map type no longer shows Birds Eye – instead, if you right-click on a point on the map, you clip_image005may get the option of viewing in bird’s eye view, and maybe streetside (though don’t expect the same level of coverage of Google Street View, at least if you’re outside the US).

If the bird’s eye option is greyed out, that means the view is not available – or no longer available, as out of date maps have been temporarily removed from the platform. Or at least removed from being easily found – if the missing area is on a boundary between data sets that are being updated, it may be possible to find a nearby place, and scroll across to find the view you’d originally searched for.

clip_image007Shortly after going live, UK users found a temporary problem – clicking on the bird’s eye view (when it’s not greyed out) didn’t do anything if your locale was “United Kingdom”.

If selecting the view doesn’t work for you, try clicking on the “hamburger” menu in the top right, and change the country setting to be United States… whereupon the bird’s eye will work, but the Ordnance Survey view will disappear.

So for a while, it looks like Brits needed to say Bye Bye, to the Bird’s Eye (for your morning coffee, see what that last link will mean for Western civilisation). Boom Boom.

clip_image009Of course, another aerial view option would be to turn to Google Maps – one tip for using the aerial view (activate via the Satellite icon in the lower left), is that if you see older images, you may well be looking at a map layer associated with the 3D view, where the service renders the view by clip_image011“underlaying” building shapes with the images. Handy in a city, maybe, less so in the countryside.

In some cases, more recent images are available if you switch 3D off, by going to the hamburger menu in the top left; try disabling 3D to see what difference that makes to your aerial view.

Tip o’ the Week 397 – Bing visual image search

Loyal Microsoft fanbois and grrlz will doubtless use Bing as their default search engine, and many “ordinary” computer users will also stick with whatever their browser or phone apps default to. Even after years of trying, though, Bing is still very much a runner-up in the league of most-used search engines, even if arguably it’s as good or even better than the alternative. Recent stats suggest that in the US, 1/3 of all searches are handled by Bing, so it’s at least in a credible 2nd place rather than a distant irrelevance, as some detractors may say.

clip_image002Even the most persistent marketers have largely given up trying to make the verb “to Bing” catch on, and El Reg reports that Google is trying to encourage “search with Google” in a style guide for developers: ‘Don’t Google Google, Googling Google is wrong’, says Google.

Aside from the beautiful daily home-screen images, there are some neat and sometimes hidden tricks in using Bing.com to search for stuff online.

A little while back, the Bing team launched Visual Search – when you do a search and look at results in Images, click on a result to preview it, and you’ll see a small magnifying glass with a dotted line box around it, in the top left.

clip_image004Clicking on that icon will let you move and resize a box around some element of the image you’re interested in, and below, you’ll see a list of related images to the one you just selected.

clip_image006

Handy for finding out about a specific item in a picture, or a person in a photo, for example.
(Yes, it is, in fact, Kevin Turvey)

This kind of searching is a variant on another approach, where you can either point Bing at an clip_image008existing online image, or upload one that you have on your computer, and it will find similar images to that. The Visual Search UI makes it a little easier if you just want to find out about a part of the image.

Watch out for some upcoming additions to Visual Search – like the ability to recognise faces in search results, for example. Read more about that and other Bing improvements to come, here.