They are not essential, but they make code that operates in lisp more compact and pleasant to write.

In Scheme my code is littered with

  (if (null? lst)
      ;; handle empty case here
      ...)

Simply because otherwise car throws an error. This whole section is often unnecessary in CL.

But you need to handle the empty case anyway otherwise you process nils ad infinitum.

You can say

  (if lst
    ...)

if the empty list is falsy, but Scheme eventually chose to add #t and #f. Oddly #f is the only false value but #t is not the only true value.

I do prefer nil being the false value as well as the empty list, even if it makes it more awkward to distinguish between 'there is a result, but the result is an empty list' and 'there are no results'. But that has nothing to do with car and cdr in Common Lisp treating nil as though it were `(cons nil nil)'. The only value in that I can see is would be if rplaca and rplacd can do some useful things with that (so `(setf (car symbol-that-currently-points-at-nil) foo)' and `(setf (cdr stcpat) bar)' do those useful things).

An oldie: https://ashwinram.org/1986/01/28/a-short-ballad-dedicated-to...

Describes the evolution from:

  (cdr (assq key a-list))

to:

  (let ((val (assq key a-list)))
     (cond ((not (null? val)) (cdr val))
           (else nil)))

Now, which is more pleasant to read (arguably the more important question for all, but the most primitive of applications)?