AWS Glue – Can not create a Path from an empty string

1. Overview

I was receiving this error while trying to run an AWS Glue job that communicated with a DB2 11 instance:

 Exception in thread "main" java.lang.IllegalArgumentException: Can not create a Path from an empty string

As we know, AWS Glue is a fully managed ETL service that is built on an Apache Spark environment. As such, AWS Glue jobs can be written in Scala or Python (pyspark).

My Glue job was written in Python and this error didn’t really seem to tell me much. Where is there an empty string I wondered?

I searched online and found a lot of results that did not apply to me, such as this one on apache.org, where the solution is: “I degradate the spark version from 2.2.0 to 2.1.1, is resolved with the error.
So phoenix 4.11 -hbase1.20 with spark2.2.0 is not work, the compatibility is not good.”

I have limited AWS Glue knowledge at this point. However, since it is a managed service, I did not find it applicable.

Thankfully, I did manage to solve this issue.

2. Solution

Here are the steps I took to solve the Can not create a Path from an empty string error in my Glue job:

  1. Ensure you can connect to the database using software like DBeaver. Check the schema.
  2. Remove all code from Glue job that did not create connection to DB2 database and run a simple command. In my case, this command was df.printSchema()
  3. Remove connection string information for database and enter it all in again. Double check that it is 100% correct.
  4. Create a new Glue job and double check IAM permissions in AWS.
  5. Ensure drivers are available in “Security configuration, script libraries, and job parameters (optional)” portion of job creation.
  6. Ensure connection is available in Required Connections of job creation/edit job.

3. Simple Code Example for Testing Database

As I said, I chose to test the least amount of code possible to ensure database connectivity. If you are having this error, it is possible you have a database connection issue or perhaps a query issue for your table.

Here is the code I ended up using for my test:

import sys
import boto3
import json
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.dynamicframe import DynamicFrame
from awsglue.job import Job
from pyspark.sql.functions import *
from pyspark.sql.functions import col, asc

args = getResolvedOptions(sys.argv, ['JOB_NAME'])

sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args['JOB_NAME'], args)

db_username = "username"
db_password = "password!"
db_url = "jdbc:db2://12.345.67.891:50000/somedatabase"
table_name = "database.sometable"
jdbc_driver_name = "com.ibm.db2.jcc.DB2Driver"

df = glueContext.read.format("jdbc").option("driver", jdbc_driver_name).option("url", db_url).option("dbtable",
                                                                                                     table_name).option(
    "user", db_username).option("password", db_password).load()

db2_schema = df.printSchema()


There are unnecessary imports, of course. If you change this code for your needs, it will work though.

4. Conclusion

If you are receiving this mysterious error on AWS Glue:

 Exception in thread "main" java.lang.IllegalArgumentException: Can not create a Path from an empty string

You can take comfort in being able to isolate the issue to a database issue. Check the steps I mentioned above and do them methodically. I believe you will be able to learn exactly what the cause of it is.

Published by Michael

I started my website in 2017 to help people learn more about information technology in order to help them become more successful in their careers. I earned my bachelor's degree from New York University, have been working in cyber security since 2012, and live in the Washington D.C. Metro area. I have found running a blog to be one of the greatest assets in helping my career and have found it could even bring in a few extra bucks each month. Starting a blog is easy and rewarding. To keep my knowledge current, I take courses on Udemy and Treehouse. By using my links, you can get a discount on both.

Leave a comment

Your email address will not be published. Required fields are marked *