Job Framework

ThunarVFS has a job framework modelled around ThunarVfsJob. While this is very convenient, it differs quite a lot from how GIO handles complex, possibly blocking tasks. Copying a file or directory using ThunarVFS is done with

thunar_vfs_copy_file (source_path, target_path, &error)

which calls thunar_vfs_copy_files() which in turn returns a ThunarVfsTransferJob. The job is launched in a separate thread and reports back to the application using signals.

In GIO, copying a file is done with

g_file_copy_async (source_file, destination_file, G_FILE_COPY_NONE, 0, cancellable, 
                   progress_callback, progress_user_data, finish_callback, finish_user_data)

which uses the GIOScheduler job framework internally and reports back to the application using the progress and finish callbacks. (Unfortunately this function doesn't do recursive copies, so we need something on top of it).

Let's take ThunarVfsDeepCountJob as an example here.

I've written an experimental implementation of it based on GIOScheduler and GSimpleAsyncResult: deepcount.c, makefile.txt.

This turns the job into a simple function call:

static void g_file_deep_count_async (GFile                     *file,
                                     int                        io_priority,
                                     GCancellable              *cancellable,
                                     GFileCountProgressCallback progress_callback,
                                     gpointer                   progress_callback_data,
                                     GAsyncReadyCallback        callback,
                                     gpointer                   callback_data)

The progress_callback is executed in the GUI thread to avoids threading issues. This makes it easy to hook into jobs e.g. in order to ask the user whether he wants to overwrite a file. All other jobs can be implemented in the same way. However, it would be nice to reduce the amount of boilerplate code involved for similar job types.

This kind of job API is also not very object-oriented and doesn't involve objects and signals the way we're used to. Should we ever decided to move away from GIO this design might become a problem.

So what we really want is an abstraction layer on top of GIO-style jobs, implemented using GObject and GTypeInterface. We don't want to pass dozens of callbacks to complex jobs. Instead, we want jobs to have signals and we want to connect to these. Internally, we can still use GIO-style jobs based on asynchronous functions, so we don't have to worry about threading and scheduling.

Similar to ThunarVfsJob and ThunarVfsDeepCountJob we want:

A base class: ThunarJob with common signals like these “error”, “finished”, “status-message”, “ask”, “ask-replace”, “progress” and “new-files”. To create jobs, there should be a number of functions like

ThunarJob *thunar_job_new_deep_count_new (GFile *file) ThunarJob *thunar_job_new_copy_file (GFile *source, GFile *destination) ThunarJob *thunar_job_new_copy_files (GList *source_files, GList *destination_files)

Internally, these could look like this:

ThunarJob *
thunar_job_new_deep_count (GFile *file)
{
  ThunarJob *job;
  GList     *source_files = NULL;

  g_return_val_if_fail (G_IS_FILE (file), NULL);
  
  source_files = g_list_append (source_files, file);

  job = g_object_new (THUNAR_TYPE_JOB, 
                      "source-files", source_files,
                      "job-function", thunar_job_deep_count,
                      NULL);

  g_list_free (source_files);

  return job;
}

where thunar_job_deep_count is a ThunarJobFunc:

typedef gboolean (*ThunarJobFunc) (ThunarJobData *data);

and ThunarJobData is a special struct for jobs:

typedef struct
{
  GIOSchedulerJob *gio_job;
  GCancellable    *cancellable;
  ThunarJob       *job;
  GError         **error;
  GList           *source_files;
  GList           *target_files;
} ThunarJobData;

ThunarJob can create and run all jobs in the same fashion. The public API is reduced to the very minimum:

void             thunar_job_launch              (ThunarJob        *job);
void             thunar_job_cancel              (ThunarJob        *job);
void             thunar_job_emit_status_message (ThunarJobData    *data
                                                 const gchar      *message);
ThunarJobResponse thunar_job_emit_ask           (ThunarJobData    *job,
                                                 const gchar      *message,
                                                 ThunarJobResponse choices);
...

I've written a simplified sample implementation in thunar-job.c and thunar-job.h.

Here's a quick and dirty implementation of the job created with thunar_job_new_deep_count().

static gboolean
thunar_job_real_deep_count (ThunarJobData *data,
                            GFile         *file,
                            goffset       *num_files,
                            goffset       *num_bytes)
{
  GFileEnumerator *enumerator;
  GFileInfo       *info;
  GFileInfo       *child_info;
  GFile           *child;
  gboolean         success = TRUE;
  gchar           *message;
  
  g_return_val_if_fail (G_IS_FILE (file), FALSE);

  info = g_file_query_info (file, 
                            "standard::*", 
                            G_FILE_QUERY_INFO_NOFOLLOW_SYMLINKS, 
                            data->cancellable, 
                            data->error);

  if (g_cancellable_is_cancelled (data->cancellable))
    return FALSE;

  if (info == NULL)
    return FALSE;

  *num_files += 1;
  *num_bytes += g_file_info_get_size (info);

  message = g_strdup_printf ("%lld files, %.2f MB", 
                             *num_files, 
                             *num_bytes / 1024.0 / 1024.0);
  thunar_job_emit_status_message (data, message);
  g_free (message);

  if (g_file_info_get_file_type (info) == G_FILE_TYPE_DIRECTORY)
    {
      enumerator = g_file_enumerate_children (file, 
                                              "standard::*", 
                                              G_FILE_QUERY_INFO_NOFOLLOW_SYMLINKS, 
                                              data->cancellable, 
                                              data->error);
    
      if (!g_cancellable_is_cancelled (data->cancellable))
        {
          if (enumerator != NULL)
            {
              while (!g_cancellable_is_cancelled (data->cancellable) && success)
                {
                  child_info = g_file_enumerator_next_file (enumerator, 
                                                            data->cancellable, 
                                                            data->error);

                  if (g_cancellable_is_cancelled (data->cancellable))
                    break;

                  if (child_info == NULL)
                    {
                      if (*(data->error) != NULL)
                        success = FALSE;
                      break;
                    }

                  child = g_file_resolve_relative_path (file, g_file_info_get_name (child_info));
                  success = success && thunar_job_real_deep_count (data, 
                                                                   child, 
                                                                   num_files,
                                                                   num_bytes);
                  g_object_unref (child);

                  g_object_unref (child_info);
                }

              g_object_unref (enumerator);
            }
        }
    }

  g_object_unref (info);

  return !g_cancellable_is_cancelled (data->cancellable) && success;
}

static gboolean
thunar_job_deep_count (ThunarJobData *data)
{
  GFile   *file;
  gboolean success;
  goffset  num_files = 0;
  goffset  num_bytes = 0;

  g_return_val_if_fail (data != NULL, FALSE);
  g_return_val_if_fail (THUNAR_IS_JOB (data->job), FALSE);
  g_return_val_if_fail (g_list_length (data->source_files) == 1, FALSE);

  if (g_cancellable_set_error_if_cancelled (data->cancellable, data->error))
    return FALSE;

  file = g_list_first (data->source_files)->data;

  g_assert (G_IS_FILE (file));

  success = thunar_job_real_deep_count (data, file, &num_files, &num_bytes);

  if (*(data->error) != NULL && g_cancellable_is_cancelled (data->cancellable))
    {
      g_error_free (*(data->error));
      *(data->error) = NULL;
    }

  if (g_cancellable_set_error_if_cancelled (data->cancellable, data->error))
    return FALSE;
  else
    return success;
}

ThunarJob *
thunar_job_new_deep_count (GFile *file)
{
  ThunarJob *job;
  GList     *source_files = NULL;

  g_return_val_if_fail (G_IS_FILE (file), NULL);
  
  source_files = g_list_append (source_files, file);

  job = g_object_new (THUNAR_TYPE_JOB, 
                      "source-files", source_files,
                      "job-function", thunar_job_deep_count,
                      NULL);

  g_list_free (source_files);

  return job;
}

And here's another dirty non-recursive implementation of g_file_copy() based on ThunarJob (as thunar_job_new_copy_file()). It uses a simplified version of the “ask” signal to demonstrate that all kinds of signals work even from within asynchronous jobs:

static void
thunar_job_copy_file_progress (goffset        current_num_bytes,
                               goffset        total_num_bytes,
                               ThunarJobData *data)
{
  gchar *message;

  message = g_strdup_printf ("%lld / %lld bytes", 
                             current_num_bytes, 
                             total_num_bytes);
  thunar_job_emit_status_message (data, message);
  g_free (message);
}

static gboolean
thunar_job_copy_file (ThunarJobData *data)
{
  GFile   *source_file;
  GFile   *target_file;
  gboolean success;

  source_file = g_list_first (data->source_files)->data;
  target_file = g_list_first (data->target_files)->data;

  if (!g_file_copy (source_file, 
                    target_file,
                    G_FILE_COPY_NONE,
                    data->cancellable,
                    (GFileProgressCallback) thunar_job_copy_file_progress,
                    data,
                    data->error))
    {
      if ((*(data->error))->code != G_IO_ERROR_EXISTS)
        return FALSE;

      if (!thunar_job_emit_ask (data))
        return FALSE;

      g_error_free (*(data->error));
      *(data->error) = NULL;

      return g_file_copy (source_file,
                          target_file,
                          G_FILE_COPY_OVERWRITE,
                          data->cancellable,
                          (GFileProgressCallback) thunar_job_copy_file_progress,
                          data,
                          data->error);
    }
  else
    return TRUE;
}

ThunarJob *
thunar_job_new_copy_file (GFile *source_file,
                          GFile *target_file)
{
  ThunarJob *job;
  GList     *source_files = NULL;
  GList     *target_files = NULL;

  source_files = g_list_append (source_files, source_file);
  target_files = g_list_append (target_files, target_file);

  job = g_object_new (THUNAR_TYPE_JOB, 
                      "source-files", source_files,
                      "target-files", target_files,
                      "job-function", thunar_job_copy_file,
                      NULL);

  g_list_free (source_files);
  g_list_free (target_files);

  return job;
}

Of course ThunarJob can be subclassed for special purposes. All in all, we need the following classes to cover the functionality present in ThunarVFS:

• ThunarJob (base class)
• ThunarSimpleJob (derived from ThunarJob, making one-function jobs easy to create)
• ThunarDeepCountJob (derived from ThunarJob with an additional “status-ready” signal)
• ThunarTransferJob (derived from ThunarJob for copying/moving files)