Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
Home
Discussion GroupsGeneralPHPASPPerlColdFusionFlashHTML, CSS, ScriptsBrowsers

Webmaster Forum / ASP / General ASP Topics / July 2008



Tip: Looking for answers? Try searching our database.

Adodb.Stream - Problem download big files

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Juan - 03 Jul 2008 14:00 GMT
Hi!

I want to use ASP to download big files using ADODB.STREAM. It works very
fine with files smaller than 80 MB.
On the Webserver I can see that memory allocation and the process w3wp is
running. After some time (more or less 2 minutes) I get a response timeout.

Here is the code:

Server.ScriptTimeout = 30000
Response.Buffer = True

Response.Clear
Response.Expires = 0
Response.ContentType = "Download-File"
Response.AddHeader "Content-Disposition","attachment; filename=" & sfile

Set oStream = Server.CreateObject("ADODB.Stream")
oStream.Type = adTypeBinary
oStream.Open
oStream.LoadFromFile(sfile)
Response.AddHeader "Content-Length", oStream.Size  ' -- Schönheit
Response.CharSet = "UTF-8"

For i = 0 To oStream.Size
   i = i + 128000
   Response.BinaryWrite(oStream.Read(128000))
   Response.Flush

Next

oStream.Close
Set oStream = Nothing
Response.Flush
Response.End

Do I have to change something in my code - or perhaps a general setting in
IIS / the metabase?

Many thanks in advance

Juan
Anthony Jones - 03 Jul 2008 14:33 GMT
> Hi!
>
[quoted text clipped - 6 lines]
>
> Server.ScriptTimeout = 30000

30,000 seconds is a long script time out ;)

> Response.Buffer = True

Change to False, you don't want to buffer anyway (yes I know you are
flushing but this is simpler).

> Response.Clear
> Response.Expires = 0
> Response.ContentType = "Download-File"

This is not a valid content type if you don't know what the content type is
use the fallback type:  "application/octet-stream"

> Response.AddHeader "Content-Disposition","attachment; filename=" & sfile
>
[quoted text clipped - 3 lines]
> oStream.LoadFromFile(sfile)
> Response.AddHeader "Content-Length", oStream.Size  ' -- Schönheit

Don't add the Content-Length header its a duplication, ASP adds that for
you.

> Response.CharSet = "UTF-8"

How come you know the character encoding but don't know the content type?
Consider deleting this line or setting the content-type to the correct text
type.

> For i = 0 To oStream.Size
>     i = i + 128000
>     Response.BinaryWrite(oStream.Read(128000))
>     Response.Flush

No need for a flush when buffering is off.

> Next
>
> oStream.Close
> Set oStream = Nothing
> Response.Flush
> Response.End

Again no need for a flush and .End is draconian, only use it if you know
something bad will happen if you don't.  Even then design out the something
bad rather than use .End.

> Do I have to change something in my code - or perhaps a general setting in
> IIS / the metabase?

The response timeout is a clientside thing however normally response
timeouts related to the time it takes to get the first chunk of data not the
overall send time.

Signature

Anthony Jones - MVP ASP/ASP.NET

Daniel Crichton - 03 Jul 2008 17:14 GMT
Anthony wrote  on Thu, 3 Jul 2008 14:33:27 +0100:

>> Hi!

>> I want to use ASP to download big files using ADODB.STREAM. It works
>> very fine with files smaller than 80 MB.
>> On the Webserver I can see that memory allocation and the process
>> w3wp is running. After some time (more or less 2 minutes) I get a
>> response
> timeout.

>> Set oStream = Server.CreateObject("ADODB.Stream")
>> oStream.Type = adTypeBinary oStream.Open oStream.LoadFromFile(sfile)
>> Response.AddHeader "Content-Length", oStream.Size  ' -- Schönheit

> Don't add the Content-Length header its a duplication, ASP adds that
> for you.

If the stream is being written out in chunks (which it is in the For Next
loop), how does the ASP engine know what the final size will be?

Signature

Dan

Daniel Crichton - 03 Jul 2008 17:31 GMT
Daniel wrote to Anthony Jones on Thu, 3 Jul 2008 17:14:31 +0100:

> Anthony wrote  on Thu, 3 Jul 2008 14:33:27 +0100:

>>> Hi!

>>> I want to use ASP to download big files using ADODB.STREAM. It works
>>> very fine with files smaller than 80 MB.
>>> On the Webserver I can see that memory allocation and the process
>>> w3wp is running. After some time (more or less 2 minutes) I get a
>>> response
>> timeout.

>>> Set oStream = Server.CreateObject("ADODB.Stream")
>>> oStream.Type = adTypeBinary oStream.Open oStream.LoadFromFile(sfile)
>>> Response.AddHeader "Content-Length", oStream.Size  ' -- Schönheit

>> Don't add the Content-Length header its a duplication, ASP adds that
>> for you.

> If the stream is being written out in chunks (which it is in the For
> Next  loop), how does the ASP engine know what the final size will be?

Actually, I've just realised that the code I use myself doesn't set the
Content-Length header either. ASP doesn't add the Content-Length header
itself either as it has no idea what the file size is when it starts
outputting it, here's an example of the ASP headers from a download from my application:

HTTP/1.1 200 OK
Cache-Control: private
Date: Thu, 03 Jul 2008 16:26:53 GMT
Transfer-Encoding: chunked
Content-Type: application/pdf
Server: Microsoft-IIS/6.0
Content-Disposition: filename=Certifications_at_a_Glance.pdf;

This just means that you don't get the status bar in IE, for instance,
showing the download progress because the final size is not known. Adding
the Content-Length header allows IE (and any other browser that will show a
status bar) to give a progress with a final size value. I tested adding the
content length header to my own application and the headers changed to:

HTTP/1.1 200 OK
Cache-Control: private
Date: Thu, 03 Jul 2008 16:29:46 GMT
Transfer-Encoding: chunked
Content-Length: 1890660
Content-Type: application/pdf
Server: Microsoft-IIS/6.0
Content-Disposition: filename=Certifications_at_a_Glance.pdf;

and appeared to have no detrimental effect on the download.

Signature

Dan

Anthony Jones - 03 Jul 2008 22:17 GMT
> Daniel wrote to Anthony Jones on Thu, 3 Jul 2008 17:14:31 +0100:
>
[quoted text clipped - 50 lines]
>
> and appeared to have no detrimental effect on the download.

You're right it doesn't have any detrimental affect on IE in fact it serves
a useful purpose.  However it is a strictly a breach of the protocol See
http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4 para 3.
However the protocol does then indicate that if it is Content-Length is
present it should be ignored in this case.

Signature

Anthony Jones - MVP ASP/ASP.NET

Daniel Crichton - 03 Jul 2008 17:22 GMT
Juan wrote  on Thu, 3 Jul 2008 06:00:01 -0700:

> Hi!

> I want to use ASP to download big files using ADODB.STREAM. It works
> very  fine with files smaller than 80 MB.
> On the Webserver I can see that memory allocation and the process w3wp
> is  running. After some time (more or less 2 minutes) I get a response
> timeout.

> Here is the code:

> Server.ScriptTimeout = 30000
> Response.Buffer = True

> Response.Clear
> Response.Expires = 0
> Response.ContentType = "Download-File"
> Response.AddHeader "Content-Disposition","attachment; filename=" &
> sfile

> Set oStream = Server.CreateObject("ADODB.Stream")
> oStream.Type = adTypeBinary  oStream.Open  oStream.LoadFromFile(sfile)
> Response.AddHeader "Content-Length", oStream.Size  ' -- Schönheit
> Response.CharSet = "UTF-8"

> For i = 0 To oStream.Size
>       i = i + 128000
>      Response.BinaryWrite(oStream.Read(128000))
>     Response.Flush
> Next

This looks odd - you're looping from 0 to the size of the stream, yet you're
pushing out 128000 bytes at a time. I think you should be using

For i = 0 To oStream.Size Step 128000

Also, in your loop what happens when you reach the end of the stream?

This is how I handle download from one of my applications

           Set oStream = Server.CreateObject("ADODB.Stream")
           Call oStream.Open()
           oStream.Type = 1
           call oStream.LoadFromFile(strFilename)

           iDownload = 1
           If lcase(right(strfilename,4)) = ".pdf" then
               Response.ContentType = "application/pdf"
           else
               Response.ContentType = "application/octet-stream"
           end if
           Response.AddHeader "Content-Disposition", "filename=" &
strfilename & ";"

           Response.Buffer = False

           'stream out the file in chunks
           Do While Not (oStream.EOS)
               Response.BinaryWrite oStream.Read(1024 * 256)
           Loop

           oStream.Close
           Set oStream = Nothing

this reads the file into the stream (just like you already have), sets an
appropriate ContentType (as Anthony points out the one you use is not a
recognised MIME type), turns off buffering, and then writes out the stream
in 256kB chunks until there is nothing left to write to out (using
oStream.EOS to determine if the stream is at the end).

You really do need buffering turned off - it's entirely possible that the
80MB file is going over the defined ASP buffer limit and so you're getting
an ASP error thrown into the data, and that causes what appears to be a
timeout but is actually due to ASP terminating the script because it has
errored. So far with the above code I've had no trouble with files larger
than the defined ASP buffer limit, but I haven't tried it with anything as
large as your file.

Signature

Dan

Daniel Crichton - 03 Jul 2008 17:37 GMT
Daniel wrote to Juan on Thu, 3 Jul 2008 17:22:58 +0100:

> Juan wrote  on Thu, 3 Jul 2008 06:00:01 -0700:

>> Hi!

>> I want to use ASP to download big files using ADODB.STREAM. It works
>> very  fine with files smaller than 80 MB.
>> On the Webserver I can see that memory allocation and the process
>> w3wp is  running. After some time (more or less 2 minutes) I get a
>> response timeout.

>> Here is the code:

>> Server.ScriptTimeout = 30000
>> Response.Buffer = True

>> Response.Clear
>> Response.Expires = 0
>> Response.ContentType = "Download-File"
>> Response.AddHeader "Content-Disposition","attachment; filename=" &
>> sfile

>> Set oStream = Server.CreateObject("ADODB.Stream")
>> oStream.Type = adTypeBinary  oStream.Open
>> oStream.LoadFromFile(sfile)
>> Response.AddHeader "Content-Length", oStream.Size  ' -- Schönheit
>> Response.CharSet = "UTF-8"

>> For i = 0 To oStream.Size       i = i + 128000
>>      Response.BinaryWrite(oStream.Read(128000))
>>     Response.Flush
>> Next

> This looks odd - you're looping from 0 to the size of the stream, yet
> you're  pushing out 128000 bytes at a time. I think you should be using

I actually had another idea as to why it might be "timing out" - for 80MB of
data, your loop is running 83886080 times (80 * 1024 * 1024). If you are
using 128kB chunks, it only needs 666 loops to send 80MB. Check your event
logs and see if you have any ASP or IIS events pointing to the process being
terminated.

Signature

Dan

Anthony Jones - 03 Jul 2008 22:08 GMT
> Daniel wrote to Juan on Thu, 3 Jul 2008 17:22:58 +0100:
>
[quoted text clipped - 38 lines]
> logs and see if you have any ASP or IIS events pointing to the process being
> terminated.

The loop is incrementing 128001 every iteration so potentially it may not
send all the file

Signature

Anthony Jones - MVP ASP/ASP.NET

Anthony Jones - 03 Jul 2008 22:20 GMT
> Hi!
>
[quoted text clipped - 36 lines]
>
> Many thanks in advance

FYI this is the routine I use for sending a file to response.

Sub SendFileToResponse(FilePath, FileName)

Const clChunkSize = 1048576 ' 1MB

Dim oStream, i
Response.Buffer = False

Response.ContentType = "application/octet-stream"
Response.AddHeader "Content-Disposition", _
 "attachment; Filename=" & FileName

Set oStream = Server.CreateObject("ADODB.Stream")
oStream.Type = 1 ' Binary
oStream.Open
oStream.LoadFromFile FilePath

For i = 1 To oStream.Size \ clChunkSize
  Response.BinaryWrite oStream.Read(clChunkSize)
Next
If (oStream.Size Mod clChunkSize) <> 0 Then
  Response.BinaryWrite oStream.Read(oStream.Size Mod clChunkSize)
End If
oStream.Close

End Sub

1MB buffer is a bit on the big side though 128K is a good choice.

Signature

Anthony Jones - MVP ASP/ASP.NET

Evertjan. - 04 Jul 2008 07:59 GMT
Anthony Jones wrote on 03 jul 2008 in
microsoft.public.inetserver.asp.general:

> FYI this is the routine I use for sending a file to response.
>
[quoted text clipped - 20 lines]
>    Response.BinaryWrite oStream.Read(oStream.Size Mod clChunkSize)
>  End If

Would an error be given if an empty chunk or is sent?
And if the requested chunk size is larger than the remainder?

If not I would do:

For i = 0 To oStream.Size \ clChunkSize
   Response.BinaryWrite oStream.Read(clChunkSize)
Next

>  oStream.Close
>
> End Sub
>
> 1MB buffer is a bit on the big side though 128K is a good choice.

Signature

Evertjan.
The Netherlands.
(Please change the x'es to dots in my emailaddress)

Alhambra Eidos Kiquenet - 08 Jul 2008 14:04 GMT
Mister, any solution about it ??

any sample code that works fine ?? please

thanks in advance...

Signature

http://www.alhambra-eidos.es/web2005/index.html
www.kiquenet.net
http://www.setbb.com/putainformatica/viewtopic.php?p=843
www.trabajobasura.com/solusoft

> Hi!
>
[quoted text clipped - 38 lines]
>
> Juan
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.