FB Messenger Image Scraper

Want to bulk download Images from Facebook Messenger in high resolution? Read on, or scroll right through to the code. This simple Image Scraper for Messenger is available to all.

The Story

Well, it’s been quite a long time since my last blog post. Life has been busy, with full of ups and downs. But I digress, that’s not how you stumbled upon this post most likely.

So, I’ll just cut right to the chase, and right down to the story.

A few weeks ago, an old friend reached out and asked if I knew how to download images from Facebook Messenger. His girlfriend and himself had recently attended a wedding ceremony. That evening, she had exchanged hundreds of photos with him. A few days later, he encountered this very problem.

At the time of writing this post, Messenger does not make it easy to bulk download high-resolution images shared over each chat.

I was interested. Having not done any Python programming in a while, I figured I’d look into it, once I had some time. Several weeks later, and here we are.

The (brief) Research

Given that I wanted to do this in Python, I needed to find a Python API that allowed me to easily interact with Messenger.

A quick search yielded a few results, but I was quickly seduced by fbchat.

The Implementation

After spending a bit of time re-configuring my Python environment (Python 3.7, pip, PyCharm, venv, etc.) and creating a new repository on GitHub, I got to work.

Thankfully Python pretty much reads itself, so I encourage you to simply read through the following if you want to understand how it works. Building the Image Scraper script was, overall, pretty easy.

GitHub (most up-to-date): https://github.com/cbdelavenne/fb-messenger-media-scraper

import os
import requests
import time
import uuid
import configparser
import datetime

from fbchat import Client, ImageAttachment
from fbchat import FBchatException
from pathlib import Path

politeness_index = 0.5  # ;)
epoch = datetime.datetime(1970, 1, 1)

def download_file_from_url(url, target_path):
    Download image from a given URL to a specified target path.

    :param url: URL of file to download
    :param target_path: Local target path to save the file
    :type url: str
    :type target_path: str
    if url is not None:
        r = requests.get(url)
        with open(target_path, 'wb') as f:
            print('\tDownloading image to {path}'.format(path=target_path))

def convert_date_to_unix_ms(date, as_int=True):
    Convert a given date string to epoch (int in milliseconds)

    :param date: Date string (preferred format %Y-%m-%d)
    :param as_int: Return unix timestamp as an integer value, instead of a float
    :type date: str
    :type as_int: int
    :return: int
        dt = datetime.datetime.strptime(date, '%Y-%m-%d')
        res = ((dt - epoch).total_seconds() * 1000)  # convert to milliseconds

        return int(res) if as_int else res
    except ValueError:
        return None

if __name__ == '__main__':
    config_path = Path('.') / 'config.ini'
    if os.path.exists(config_path) is False:
        raise Exception("Please create config.ini under this script's current directory")

    # Load config file
    config = configparser.ConfigParser()

    download_path = config.get('Download', 'path')
    if os.path.exists(download_path) is False:
        raise Exception("The path specified in download_path does not exist ({path}). Please specify a valid path in "

    # Initialize FB Client
    fb_email = config.get('Credentials', 'email')
    fb_pw = config.get('Credentials', 'password')
    fb_client = Client(fb_email, fb_pw)

    # Search for latest threads
    thread_search_limit = int(config.get('Threads', 'search_limit'))
    thread_search_before = convert_date_to_unix_ms(config.get('Threads', 'before_date'))

    if thread_search_before is not None:
        threads = fb_client.fetchThreadList(limit=thread_search_limit, before=thread_search_before)
        threads = fb_client.fetchThreadList(limit=thread_search_limit)

    # Find correct thread for given user URL
    my_thread = None
    for thread in threads:
        if thread.url == config.get('Friend', 'url'):
            my_thread = thread

    # Get Messages for my_thread
    if my_thread is not None:
        message_search_limit = int(config.get('Messages', 'search_limit'))
        message_search_before = convert_date_to_unix_ms(config.get('Messages', 'before_date'))

        if message_search_before is not None:
            messages = fb_client.fetchThreadMessages(my_thread.uid, limit=message_search_limit,
            messages = fb_client.fetchThreadMessages(my_thread.uid, limit=message_search_limit)

        # Extract Image attachments' full-sized image signed URLs (along with their original file extension)
        full_images = []

        sender_id = None
        if config.getboolean('Media', 'sender_only'):
            sender_id = my_thread.uid

        for message in messages:
            if len(message.attachments) > 0:
                if (sender_id is None) or (sender_id == message.author):
                    for attachment in message.attachments:
                        if isinstance(attachment, ImageAttachment):
                                    'extension': attachment.original_extension,
                                    'full_url': fb_client.fetchImageUrl(attachment.uid)
                            except FBchatException:
                                pass  # ignore errors

        # Download Full Images
        if len(full_images) > 0:
            images_count = len(full_images)

            print('Attempting to download {count} images...................\n'.format(count=images_count))

            for full_image in full_images:
                friend_name = str.lower(my_thread.name).replace(' ', '_')
                file_uid = str(uuid.uuid4())
                file_ext = full_image['extension']
                img_url = full_image['full_url']

                image_path = ''.join([download_path, '\\', 'fb-image-', friend_name, '-', file_uid, '.', file_ext])

                download_file_from_url(img_url, image_path)

                # Sleep half a second between file downloads to avoid getting flagged as a bot
            print('No images to download in the last {count} messages'.format(count=message_search_limit))
        print('Thread not found for URL provided')

Sample output:

Logging in {EMAIL}...
Login of {EMAIL} successful.
Attempting to download 2 images...................

 Downloading image to c:\\Users\\{USER}\\Downloads\fb-image-{FRIEND_NAME}-50345061-3ff9-4f0a-a6f4-1988a4259a62.png
 Downloading image to c:\\Users\\{USER}\\Downloads\fb-image-{FRIEND_NAME}-9deb01e8-eae2-4915-a28f-ac61d85bea2e.png

Some Caveats

The current version of the script comes with a few caveats:

  1. Finding the chat you’re looking for isn’t the easiest. First you need your friend’s profile page URL. Then, the fbchat API seems to be restricted to searching through 20 threads at a time.
  2. Finding the messages you’re looking for isn’t the easiest either. I’m also not sure what the query limit is for messages.
  3. The script could use some extra error handling for users that aren’t necessarily comfortable editing this code themselves.

Get any GitHub user’s email (Python)

Here’s a silly little Python script (github_email.py) I wrote to get a GitHub user’s email address based on their public event history.

It’s not bulletproof, but generally works:

from __future__ import print_function

import requests
import sys

from optparse import OptionParser

parser = OptionParser()
parser.add_option('-u', '--username', action='store', type='string', dest='username')
(options, args) = parser.parse_args()

gh_api_url = 'https://api.github.com/users/{username}/events/public'.format(username=options.username)

r = requests.get(gh_api_url)

gh_public_events = r.json()

if isinstance(gh_public_events, dict):
    if gh_public_events.get('message') and gh_public_events.get('message') == 'Not Found':
        print('User was not found!')

fullname = 'N/A'
email = 'N/A'

for event in gh_public_events:
    if event.get('payload').get('commits'):
        commits = event.get('payload').get('commits')
        for commit in commits:
            if commit.get('author'):
                fullname = commit.get('author').get('name')
                email = commit.get('author').get('email')

print('Full Name: {name}'.format(name=fullname))
print('email: {email}'.format(email=email))

You can simply save this script as github_email.py (for example).

Also, do note, it uses Python 3, a deprecated standard library (optparse has been replaced by argparse) and one third party library, requests.

Usage Output

Usage: github_email.py [options]

  -h, --help            show this help message and exit
  -u USERNAME, --username=USERNAME

Expected Results

If the user exists and has public events:

$ github_email.py -u jbarnette
Full Name: John Barnette
email: jbarnette@github.com

If the user exists, but has no public events, you’ll get:

$ github_email.py -u test
Full Name: N/A
email: N/A

If the user does not exist, you’ll get:

$ github_email.py -u iou18y23123
User was not found!

And that’s how you get (almost) any GitHub user’s email!
It’s useful if you want to reach out to another contributor via email.


The script is inspired from this blog post: How to Find Almost Any GitHub User’s Email Address

If you rather do this process manually, follow the instructions in the link right above.