Why does using defpackage result in a NAME-CONFLICT?

Question

So I'm practicing lisp with Project Euler, and I'm collecting little utility functions into a separate file to reduce duplication, and I expect that it might get pretty huge, so I've gone ahead and made a package definition. Here is a condensed version of the file that still illustrates my problem:

(defpackage :euler-util
    (:use :common-lisp)
    (:export :divisible-by))

(in-package :euler-util)

(defun divisible-by (x y)
    (equal 0 (mod x y)))

I can compile and load (C-c C-k) that file into SLIME without any warnings or errors. Now, when I go to use it in another file, I do the following:

(load "util.lisp")
(use-package :euler-util)

(defun euler-1 ()
    (loop for i from 3 to 999 when (or (divisible-by i 3) (divisible-by i 5)) sum i))

When I try to compile and load THAT file, I get an error like this:

"USE-PACKAGE # causes name-conflicts in # between the following symbols: EULER-UTILS:DIVISIBLE-BY, COMMON-LISP-USER::DIVISIBLE-BY [Condition of type NAME-CONFLICT]"

Why is this symbol appearing in the COMMON-LISP-USER package, and how do I stop it?

Answer 1

Note: I wrote this answer in response to my initial understanding of the problem, where I though that the image was still "dirty" from some previous code. That's a common case of these sorts of symptoms. This case, however, arises from the fact that the effects of use-package don't occur at compile time, but rather at load time. Thus, when I load the file below, there's no problem, but in compiling it as OP was, there's a reference to cl-user::divisble-by. I'm leaving this answer here because people with this problem are still likely to find it, but it's not exactly the same problem that OP is having.

Your code:

I've made local copies of your code in util.lisp and file2.lisp:

$ cat util.lisp 
(defpackage :euler-util
    (:use :common-lisp)
    (:export :divisible-by))

(in-package :euler-util)

(defun divisible-by (x y)
    (equal 0 (mod x y)))

$ cat file2.lisp 
(load "util.lisp")
(use-package :euler-util)

(defun euler-1 ()
    (loop for i from 3 to 999 when (or (divisible-by i 3) (divisible-by i 5)) sum i))

Reproducing the problem

This problem is easy to reproduce: if you've somehow interned the a symbol named "DIVISIBLE-BY" into the CL-USER package, and later try to use a package that exports a symbol with the same name, you'll have trouble:

$ sbcl --noinform
* (defun divisible-by (&rest args) (declare (ignore args)))

DIVISIBLE-BY
* (load "file2")

debugger invoked on a NAME-CONFLICT in thread #<THREAD "initial thread" RUNNING
                                                 {1002979041}>:
  USE-PACKAGE #<PACKAGE "EULER-UTIL"> causes name-conflicts in
  #<PACKAGE "COMMON-LISP-USER"> between the following symbols:
    EULER-UTIL:DIVISIBLE-BY, COMMON-LISP-USER::DIVISIBLE-BY
See also:
  The ANSI Standard, Section 11.1.1.2.5

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RESOLVE-CONFLICT] Resolve conflict.
  1: [RETRY           ] Retry EVAL of current toplevel form.
  2: [CONTINUE        ] Ignore error and continue loading file "…/file2.lisp".
  3: [ABORT           ] Abort loading file "…/file2.lisp".
  4:                    Exit debugger, returning to top level.

(NAME-CONFLICT
 #<PACKAGE "COMMON-LISP-USER">
 USE-PACKAGE
 #<PACKAGE "EULER-UTIL">
 EULER-UTIL:DIVISIBLE-BY
 DIVISIBLE-BY)

Resolving the conflict

If this happens, you can resolve it manually. I'm using SBCL on the command line here, but you should (I hope) get similar debugger options in SLIME. I'm pushing a lot into the command line for brevity. The points here are that: --noinform keeps the banner from being printed; --eval "'divisible-by" makes sure that there's a symbol cl-user:divisible-by; and --load file2.lisp loads your file. That is, we've recreated the problem on the command line so that we can focus on fixing the problem.

$ sbcl --noinform --eval "'divisible-by" --load file2.lisp 

debugger invoked on a NAME-CONFLICT in thread #<THREAD "initial thread" RUNNING
                                                 {1002979311}>:
  USE-PACKAGE #<PACKAGE "EULER-UTIL"> causes name-conflicts in
  #<PACKAGE "COMMON-LISP-USER"> between the following symbols:
    EULER-UTIL:DIVISIBLE-BY, COMMON-LISP-USER::DIVISIBLE-BY
See also:
  The ANSI Standard, Section 11.1.1.2.5

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RESOLVE-CONFLICT] Resolve conflict.
  1: [RETRY           ] Retry EVAL of current toplevel form.
  2: [CONTINUE        ] Ignore error and continue loading file "/home/taylorj/tmp/package-issue/file2.lisp".
  3: [ABORT           ] Abort loading file "/home/taylorj/tmp/package-issue/file2.lisp".
  4:                    Ignore runtime option --load "file2.lisp".
  5:                    Skip rest of --eval and --load options.
  6:                    Skip to toplevel READ/EVAL/PRINT loop.
  7: [QUIT            ] Quit SBCL (calling #'QUIT, killing the process).

(NAME-CONFLICT
 #<PACKAGE "COMMON-LISP-USER">
 USE-PACKAGE
 #<PACKAGE "EULER-UTIL">
 EULER-UTIL:DIVISIBLE-BY
 DIVISIBLE-BY)

Now you can type 0 if you want to resolve the conflict:

0] 0

Select a symbol to be made accessible in package COMMON-LISP-USER:
  1. EULER-UTIL:DIVISIBLE-BY
  2. COMMON-LISP-USER::DIVISIBLE-BY

Enter an integer (between 1 and 2): 1

By making euler-util:divisible-by the symbol accessible in cl-user, your code will work as you expect:

* (euler-1)

233168

With a fresh image, there's no problem.

While you might not have caused the problem in the same way that I did above, it's probably something similar. It appears that you're loading code in a non-fresh environment (e.g., where you've already entered some code that caused the reader to intern "DIVISIBLE-BY" in the CL-USER package. With a fresh Lisp, your code loads just fine:

$ sbcl --load file2.lisp --noinform 
* (euler-1)

233168

Answer 2

compile-file reads all the forms in the file, and only executes forms that expand into the appropriate eval-when expressions. In your example above, the util.lisp file is not loaded until after the referencing file is already read (interning all its symbols in cl-user). use-package is similarly a plain function call that is not evaluated until load time, and that's when you're asking it to make two distinct symbols by the same name accessible in the current package.

One option is to put the load and use-package statements in an eval-when form, e.g.

(eval-when (:compile-toplevel :load-toplevel :execute)
  (load "util.lisp")
  (use-package :euler))

I think it would be better to define a new package for your project, and put it at the top of your file:

(defpackage #:euler
  (:use #:cl #:euler-util))

(in-package #:euler)

Those statements are automatically evaluated at compile time, so there's no need for eval-when.

Experienced CL authors tend to avoid this problem by defining packages in a specific order, putting in-package forms in every source file, and by using a system definition facility to compile and load files in the right order.

I use ASDF as a system definition facility. A simple system file could look like this:

;;;; my-project.asd

(asdf:defsystem my-project
  :serial t
  :components ((:file "util")
               (:file "my-project")))

If you put that file somewhere ASDF knows about, (asdf:load-system "my-project") will then compile and load the files in the specified order.

If you use Quicklisp, one easy way to do it is put the project's directory in ~/quicklisp/local-projects/ and then use (ql:quickload "my-project") to automatically load it and its dependencies.