Upload file with form

[apr3vau]

Hello!
I'm a noob of clog. I really love its simple and powerful, but when I tried to use clog to upload file to my server through form, I got some trouble.
I tried to use post method, just like the way in tutorial 17, and with encoding of "multipart/form-data". but after I submit the form, the form-post-data is still (("undefined") . nil) . I also tried to use post-parameter in hunchentoot, but it tell me hunchentoot:*request* is unbound and it is difficult for me to deal with it.
If anyone can give me a direction? Thanks! :)

[aykaramba]

Any chance you can post your code to a code repo and link to it? I would love to learn how to do this as well.

Sorry for the non answer!

[apr3vau]

Sorry I have no way to get it done but to start a new site with Hunchentoot and create a iframe in Clog... That's not very well :(

[aykaramba]

Did you figure this out? If yes, can you post the functions you used to process the form data and store it?

I am interested in learning how this is done.

[rabbibotton]

Can you submit this as a bug. I am not sure if added the code for file upload (post for sure implemented). This way gets in my queue to do soon.

[rabbibotton]

I have added file upload support in latest commit. See the updated tutorial 17 and source/clog-form.lisp

[aykaramba]

A quick follow up on uploading files to the server:

1. I am building a simple gallery application where users can upload images and the gallery just displays the last 10 images or something like that.

2. I have read 17-tutorial.lisp, source/clog-form.lisp and a ton of documentation online for writing to files but I am not fully understanding how to grab the data from the form-multipart-data symbol(?).

3. I have learned that I should probably be using with-open-file as opposed to using open and here is a rewritten on-page4 function from 17-tutorial.lisp that does something:

(defun on-page4 (body)

(with-open-file (my-file "~/name.nfo" :direction :output :if-exists :supersede)

   (format my-file "testing to see if it writes to text file")

   (let ((stream my-file)
       (data (form-multipart-data body)))
       (format stream "~&Data:~12T~A~%" data)
       (delete-multipart-data body))))

4. The above writes successfully to the text file the following (note: 000000000000.nfo is just a plain text file used for testing to upload using the 17-tutorial web form):

testing to see if it writes to text file
Data: ((filename #<VECTOR-INPUT-STREAM {1002D33C43}> 000000000000.nfo
text/x-nfo))

All of that is correct.

QUESTION: What is the correct way to get data from the form and save it to a file?

QUESTION: How to check if the files is binary or not? Do we need to care, can we just dump the raw data to file and just name it whatever the uploader is uploading? It looks like we should be using Flexistreams for this, is that correct?

QUESTION: Tangential to this, does anyone have any general hints on where to look for examples on how to check / recognize filetypes beyond parsing the extension?

Any hints on the above questions are appreciated.

[rabbibotton]

I'll answer rest later, but another example of using form upload is in the builder - this takes the input of a panel that has an upload form and creates a new page with the image converted:

(defun on-convert-image (body)
  (let ((params (form-multipart-data body)))
    (create-div body :content params)
    (destructuring-bind (stream fname content-type)
        (form-data-item params "filename")
      (create-div body :content (format nil "filename = ~A - (contents printed in REPL)" fname))
      (let ((s        (flexi-streams:make-flexi-stream stream))
            (pic-data ""))
        (setf pic-data (format nil "data:~A;base64,~A" content-type
                               (with-output-to-string (out)
                                 (s-base64:encode-base64 s out))))
        (create-img body :url-src pic-data)
        (create-br body)
        (create-div body :content "User the following as a url source:")
        (set-geometry (create-text-area body :value pic-data) :width 500 :height 400)
        (create-br body)
        (create-div body :content (format nil "For example:<br>(create-img body :url-src \"~A\")" pic-data))))))
```

[rabbibotton]

So here is an example to output a binary file - content-type will also tell you the content type :)

  (set-on-new-window
   (lambda (body)
     (let ((params (form-multipart-data body)))
       (create-div body :content params)
       (destructuring-bind (stream fname content-type)
           (form-data-item params "file-to-upload")
         (create-div body :content (format nil "filename = ~A - ~A (contents printed in REPL)" fname content-type))
         (with-open-file (os fname :direction :output :if-exists :rename :element-type '(unsigned-byte 8))
           (let ((s (flexi-streams:make-flexi-stream stream)))
             (loop for byte = (read-byte s nil)
                   while byte
                   do (write-byte byte os))))))
       (delete-multipart-data body))
   :path "/upload")

[rabbibotton]

Did that work out for you?

[aykaramba]

Thank you kindly for the follow up Rabbi!

Please keep in mind that I am learning Common Lisp at the same time as CLOG. The biggest challenge was learning destructuring-bind and flexi-streams. I am starting to kind of understand what is going on, but not quite there yet, I need to study more.

For 17-tutorial.lisp I took the binary example above, chopped out all the unnecessary bits and rewrote on-page4 function like this:

(defun on-page4 (body)
  (let ((params (form-multipart-data body)))
       (destructuring-bind (stream fname content-type)
	                   (form-data-item params "filename")
	                   (with-open-file (os "~/file.jpg" :direction :output :if-exists :rename :element-type '(unsigned-byte 8))
		                 (let ((s (flexi-streams:make-flexi-stream stream)))
			              (loop for byte = (read-byte s nil)
			                 while byte
			                 do (write-byte byte os)))))))```

This actually works, and I think I understand why. I now have to add the (delete-multipart-data body) form back in and then get it to save using paths.

Did your examples, help? Yup! Did I make a mockery of the code? Yes, I did.

If I get this working right I may contribute the code to tutorial 17.

Thank you kindly for the help and the hints. There would be no chance I could have figured that out without the example. If you have any suggestions or critiques, they are always appreciated.

[aykaramba]

With apologies to everyone for the python style indentation (just learning, I need to see where the forms start and end to reason about them), here is the finished working upload function for 17-tutorial.lisp:

(defun on-page4 (body)
  (let ((params (form-multipart-data body)))
       (destructuring-bind (stream fname content-type)
	                   (form-data-item params "filename")
	                   (with-open-file (os
					       (make-pathname
					          :directory '(:relative "upload")
					          :name fname)
					    :direction :output
					    :if-exists :rename
					    :element-type '(unsigned-byte 8))
		                            (let ((s (flexi-streams:make-flexi-stream stream)))
			                        (loop for byte = (read-byte s nil)
			                          while byte
			                          do (write-byte byte os))))))
  (delete-multipart-data body))

It uploads the file to a folder called "upload" with a relative path. I will add labels and such in a bit as it redirects to a page in the tutorial.

I hope this is helpful to other noobs. The cleanest reference for destructuring-bind and make-pathname are on https://gigamonkeys.com/book/. Flexi-streams, that's a google and a half.