this post was submitted on 28 Oct 2023
1 points (100.0% liked)

Lisp

52 readers
3 users here now

founded 1 year ago
MODERATORS
 

Hello guys,

I have a question regarding the nested backquotes in macros. I wrote a macro, which creates lexical bindings for "port:ip" values:

(defun mkstr (&rest args)
  (with-output-to-string (s)
    (dolist (a args) (princ a s))))

(defun mksymb (&rest args)
  (values (intern (string-upcase (apply #'mkstr args)))))

;; my macro
(defmacro with-free-ports (start end &body body)
  (let ((range (loop for port from start to end collect (format NIL "127.0.0.1:~a" port)))
	(n 0))
    `(let ,(mapcar #'(lambda (p) `(,(mksymb "PORT-" (incf n)) ,p)) range)
       (progn ,@body))))

One sets a range of ports on localhost and these ports are bound to symbols port-1, port-2, etc..

(with-free-ports 1 3 port-1) ;; => "127.0.0.1:1"

This works fine if the start or end parameters are given as values. But if they are variables. which must be evaluated, this macro doesn't work:

(let ((start 1))
  (with-free-ports start 3 port-1)) ;; error

In order to fix it, I made the let- bindings a part of the macro-expansion:

(defmacro with-free-ports (start end &body body)
  `(let ((range (loop for port from ,start to ,end collect (format NIL "127.0.0.1:~a" port)))
	(n 0))
     `(let ,(mapcar #'(lambda (p) `(,(mksymb "PORT-" (incf n)) ,p)) range)
	       (progn ,@body))))

but get a compilation warning that the body is never used. I assume this is because of the inner backquote.

To evaluate ,@body inside the inner backquote, I use one more comma, and the macro compiles without warnings:

(defmacro with-free-ports (start end &body body)
  `(let ((range (loop for port from ,start to ,end collect (format NIL "127.0.0.1:~a" port)))
	(n 0))
     `(let ,(mapcar #'(lambda (p) `(,(mksymb "PORT-" (incf n)) ,p)) range)
	       (progn ,,@body)))) ;; one more comma here

But it doesn't work:

(let ((start 1))
  (with-free-ports start 3 port-1)) ;; error: port-1 is unbound

because with this ,,@body I evaluate port-1: (progn ,port-1) and this triggers the error.

I would appreciate if smbd can help me a bit and say what I am doing wrong.

Thank you.

top 7 comments
sorted by: hot top controversial new old
[–] lispm@alien.top 1 points 10 months ago (1 children)

If you compile code, then the compiler expands macro forms at compile-time. If you want to generate a variable number of let bindings, then the number needs to be known at compile-time.

start and end thus can't be variables at run-time.

(let ((start 1))
  (with-free-ports start 3 port-1))

If we compile above form, then at compile-time the value of start generally is unknown.

[–] xhash101@alien.top 1 points 10 months ago (1 children)

Thank you for your reply.

The compiler generally will not execute the LET and then compile the WITH-FREE-PORTS form with the new binding

Here is an example when the compiler does exactly that:

(defmacro with-free-ports (start end &body body)
  `(list ,start ,end ,@body))

(let ((start 1)
      (end 3))      
  (with-free-ports start end NIL))

So, why does it work?

[–] lispm@alien.top 1 points 10 months ago

No, the compiler does not do that. If you look into your macro here, it just puts the args start and end back in. The generated code then is executed. Remember: START and END are bound to the source forms, not computed values.

In the original macro you had the form (loop for port from start to end ...), which you tried to compute at macro-expansion time. But the values of START and END are not necessary numbers, but source forms, like variable names.

[–] tdrhq@alien.top 1 points 10 months ago

Try not to fight so much with macros. I see what you're trying to do, but it might be better to just bind a list or array of ports, instead of binding port{1-n}.

If you agree to use an array for binding, then you can use https://github.com/tdrhq/easy-macros to write this macro without using any backquotes at all. The macro definition would look something like:

(def-easy-macro with-free-ports (start end &binding ports &fn fn)
  (unwind-protect
     (fn (... construct list or array of ports ...))
    (cleanup #| this cleanup is probably the reason you're using macros in the first place |#)))

And you'll call it as:

(with-free-ports (1000 2000 port)
  (... do something with port ...))
[–] dr675r@alien.top 1 points 10 months ago (1 children)

I will preface this by saying in general I don't think the approach of interning symbols and binding them at runtime is a good idea. If you've referenced these symbols by name in the body of the form, you already knew you needed them, so why not just just use LET to create the binding? Without knowing the details of your use case it sounds like it could be an XY problem.

However, this being Common Lisp, there is a way to do most things and the special operator PROGV is used to create dynamic variable bindings. This is not a tool I reach for often, but it does occasionally come in handy. I am not proud of this:

(in-package #:cl-user)
(ql:quickload "alexandria")

(defmacro with-free-ports ((start end &key (prefix "PORT-") limit) &body body)
  (check-type limit (or null (integer 1 *)))
  (alexandria:with-gensyms (first-port last-port index vars values symbol-prefix)
    `(let ((,first-port ,start)
           (,last-port ,end)
           (,symbol-prefix ,prefix))
       (check-type ,first-port (integer 1 *))
       (check-type ,last-port (integer 1 *))
       (assert (>= ,last-port ,first-port) (,first-port ,last-port) "port range not ordered")
       (check-type ,symbol-prefix (or string symbol character))
       ,(when limit
          `(when (>= (- ,last-port ,first-port) ,limit)
             (error ,(format nil "WITH-FREE-PORTS limit (~A) exceeded" limit))))
       (loop named with-free-ports
             for ,index from ,first-port to ,last-port               
             collecting (alexandria:symbolicate ,symbol-prefix (princ-to-string ,index)) into ,vars
             collecting (format nil "127.0.0.1:~A" ,index) into ,values
             finally (return-from with-free-ports (progv ,vars ,values ,@body))))))

To use it:

(let ((start 80))
  (with-free-ports (start (+ start 5) :limit 10)
    (setf port-80 "http")
    (values port-80 port-85))) ;; => "http", "127.0.0.1:85"

I have added what I consider the bare minimum of safety checks, such as a limit on the number of symbols it will create. You also lose help from the compiler which can't warn you about unused variables and you may get warned that some of your variables have been assumed special (at least on LispWorks). Overall, I think its a bit of hack and not something I would put in production code.

It is also worth noting the following loop achieves the same thing, without the hackery and is trivial to change to a vector if you're worried about efficiency:

(let ((start 80))
  (loop for port from start to (+ start 5)
        collecting (format nil "127.0.0.1:~A" port) into ports
        finally (progn
                  (setf (first ports) "http")
                  (return (values (first ports) (sixth ports))))))
[–] xhash101@alien.top 1 points 10 months ago (1 children)

Thanks a lot!

If you've referenced these symbols by name in the body of the form, you already knew you needed them,

Yes, but I do not know how many of them. The purpose of this macro is to create a lexical environment to test a network code. Some functions take just one port as a parameter, but the others - two and more. With this macro it would be very easy to call these functions and to link their IO through the port numbers:

(with-free-ports 0 10
  (fn-1 port-1 port-2 port-3)
  (fn-2 port-1))

And with the first posted version of with-free-ports I achieved it. The problems appeared when I tried to bind start in run-time. From your reply and from the u/lispm comments I understood that this is not a good idea. What I do not understand, is how to use such macros inside another macros/functions, which can supply with-free-ports with start and end parameters ? Also, I cannot figure out, when exactly macro-expansions take place if I use many enclosed macros. Are they expanded all at once when I compile my code?

Anyway, I am very grateful for your suggestions. This helps a lot to learn CL.

[–] dr675r@alien.top 1 points 10 months ago

Macros are expanded recursively, so if the result of expanding a macro is itself a macro then the evaluator or compiler will immediately expand it again. This is why we have MACROEXPAND-1 which expands it once, and MACROEXPAND which expands a form until it is no longer a macro form. I would suggest you MACROEXPAND-1 my answer, then repeat with MACROEXPAND to see the non-macro code that actually ends up being compiled/evaluated.

You may find Section 3 of the HyperSpec helpful. The spec goes into a fair amount of detail about the semantics of both evaluation and compilation which should help.