C++Talk.NET C++ language newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  fopen / OPEN_MAX          C++Talk.NET Forum Index -> C++ Language (Moderated) View previous topic :: View next topic   Author Message Kushal Agarwal Guest Posted: Fri Sep 17, 2004 10:50 am    Post subject: fopen / OPEN_MAX Hello, I need to use fopen to open a whole bunch of files, however I discovered that I am limited in the number of simultaneous file handles that can be opened by the constant OPEN_MAX. I tried to modify this constant, but it appears to be compile time set, therefore my changes are not reflected. I was wonder if anyone knew how I can modify this constant? Additionally, does anyone also know what other functions/procedures may utilize the constant OPEN_MAX? I do realize that there are other ways to open files, but for the project I am working on it is most practical to use fopen. Thanks, Kushal [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Ulrich Eckhardt Guest Posted: Sat Sep 18, 2004 11:00 am    Post subject: Re: fopen / OPEN_MAX Kushal Agarwal wrote: Quote: I need to use fopen to open a whole bunch of files, however I discovered that I am limited in the number of simultaneous file handles that can be opened by the constant OPEN_MAX. I tried to modify this constant, but it appears to be compile time set, therefore my changes are not reflected. I was wonder if anyone knew how I can modify this constant? In short, you can't. It's a limit and limits usually have the annoying behaviour of not being mutable. However, you might be able to nonetheless. How to do that depends on your OS, its libc and maybe the phase of the moon though - you will have to ask in a group related to your system. Quote: Additionally, does anyone also know what other functions/procedures may utilize the constant OPEN_MAX? I don't think it is fopen that fails in the first place, I rather think it is open() that returns EMFILE. Reading its manpage for Linux, I see a few other functions that return a filedescriptor: socket, fifo, accept. As I read the manpage, those are not parts of C[++] but rather parts of POSIX, though. Quote: I do realize that there are other ways to open files, but for the project I am working on it is most practical to use fopen. Hmmm, it's not really on-topic here, but I can't imagine a case where I'd hit the limit of fopen(). Maybe, if you described what you are trying to achieve, someone here could suggest a better way of doing things. Uli -- Questions ? see C++-FAQ Lite: http://parashift.com/c++-faq-lite/ first ! [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Andreas Harnack Guest Posted: Sat Sep 18, 2004 2:30 pm    Post subject: Re: fopen / OPEN_MAX Hi Kushal, that restriction won't be set by the language but by the underlying operating system. I guess you won't get a compiler error but run time error. It doesn't matter by the way, how you open the file, you always need a file descriptor and that must be provided by the OS. On a Unix like system you have three options: 1. Ask your admin to increase the limit, usually a kernel parameter or something like that (your application will have to be really important before he or she is likely to do that 2. Close and reopen files as needed, you won't need all of the at the same time; however, opening a file is expensive, so perhaps you want to keep a cache of open descriptors to reduce overhead. 3. Use several processes, since the limit is a per process limit. There is a system wide limit too, but that's usually much higher. Thread's won't help since threads share process recourses, file descriptors are typically among the shared resources. I'd go for option 2. (possibly combined with 3). since it won't put a per se limit on your application and yet can use all resources it can get. That still leaves the option to provide more resources by your admin if required. Andreas "Kushal Agarwal" <kushal.agarwal (AT) gmail (DOT) com> wrote Quote: Hello, I need to use fopen to open a whole bunch of files, however I discovered that I am limited in the number of simultaneous file handles that can be opened by the constant OPEN_MAX. I tried to modify this constant, but it appears to be compile time set, therefore my changes are not reflected. I was wonder if anyone knew how I can modify this constant? Additionally, does anyone also know what other functions/procedures may utilize the constant OPEN_MAX? I do realize that there are other ways to open files, but for the project I am working on it is most practical to use fopen. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top kanze@gabi-soft.fr Guest Posted: Mon Sep 20, 2004 6:29 pm    Post subject: Re: fopen / OPEN_MAX Ulrich Eckhardt <doomster (AT) knuut (DOT) de> wrote Quote: Kushal Agarwal wrote: I need to use fopen to open a whole bunch of files, however I discovered that I am limited in the number of simultaneous file handles that can be opened by the constant OPEN_MAX. I tried to modify this constant, but it appears to be compile time set, therefore my changes are not reflected. I was wonder if anyone knew how I can modify this constant? In short, you can't. It's a limit and limits usually have the annoying behaviour of not being mutable. However, you might be able to nonetheless. How to do that depends on your OS, its libc and maybe the phase of the moon though - you will have to ask in a group related to your system. Additionally, does anyone also know what other functions/procedures may utilize the constant OPEN_MAX? I don't think it is fopen that fails in the first place, I rather think it is open() that returns EMFILE. Reading its manpage for Linux, I see a few other functions that return a filedescriptor: socket, fifo, accept. As I read the manpage, those are not parts of C[++] but rather parts of POSIX, though. It depends. I've used systems where FOPEN_MAX was limited to 19, although the kernel allowed more. On some modern systems, at least, FOPEN_MAX isn't actually the limit. For some reason, under Solaris, FOPEN_MAX is 20, although I can invoke fopen over 250 times before it fails. Quote: I do realize that there are other ways to open files, but for the project I am working on it is most practical to use fopen. Hmmm, it's not really on-topic here, but I can't imagine a case where I'd hit the limit of fopen(). Well, I don't hit it, because I don't use FILE* at all:-). But it isn't hard to imagine hitting a limit of 19. Especially considering that three (stdin, stdout and stderr) are already used. -- James Kanze GABI Software http://www.gabi-soft.fr Conseils en informatique orientée objet/ Beratung in objektorientierter Datenverarbeitung 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34 [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         C++Talk.NET Forum Index -> C++ Language (Moderated) All times are GMT Page 1 of 1   Jump to: Select a forum C/C++----------------C LanguageC Language (Moderated)C++ Language (Moderated)C++ language (comp.lang.c++)C++ language, library and standards Non-English C/C++ Newsgroups----------------C++ (German)C++ (French)  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.