* Is EXT4 the right FS for > 16TB?
@ 2010-12-19 11:52 Justin Piszcz
2010-12-19 16:35 ` Sandon Van Ness
0 siblings, 1 reply; 8+ messages in thread
From: Justin Piszcz @ 2010-12-19 11:52 UTC (permalink / raw
To: linux-kernel; +Cc: linux-ext4, Alan Piszcz
Hi,
I've read a lot of posts regarding people who setup RAID volumes of and up
to around 16TB and EXT4 is typically used.
However, in various forums, people still ask what is the correct
filesystem for > 16TB? I did read one post somewhere that stated the ext4
developers did not recommend using ext4 for very large volumes, is this
still true?
I am looking at creating a 43TB volume possibly in the near future and I
have used XFS in the past, which works well and would probably not have
any problem with it; however, I have bitten quite a number of times by XFS
bugs in the past several years, so I was curious, how does EXT4 perform on
larger volumes, e.g., 20TB?
Are there any caveats / problems?
Justin.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is EXT4 the right FS for > 16TB?
2010-12-19 11:52 Is EXT4 the right FS for > 16TB? Justin Piszcz
@ 2010-12-19 16:35 ` Sandon Van Ness
2010-12-19 16:53 ` Justin Piszcz
0 siblings, 1 reply; 8+ messages in thread
From: Sandon Van Ness @ 2010-12-19 16:35 UTC (permalink / raw
To: Justin Piszcz; +Cc: linux-kernel, linux-ext4, Alan Piszcz
Was it me (houkouonchi) on hard forum? I asked if > 16 TiB support was
considered stable on here a while back:
Is >16TB support considered stable?
This was 6 months ago so maybe things have changed. The thread:
http://kerneltrap.org/mailarchive/linux-ext4/2010/5/28/6884603/thread
Luckily JFS fixed there userland utilities bug of not being able to
handle > 32TiB very shortly after this and I ended up going that route
and I have yet to have any data loss or problems on my JFS volume:
root@dekabutsu: 08:32 AM :~# df -H /data
Filesystem Size Used Avail Use% Mounted on
/dev/sdd1 36T 22T 15T 61% /data
root@dekabutsu: 08:32 AM :~#
At work with our hundreds/thousands of servers we will likely be going
ext4 as we wont be using it on >16 TiB. I think its a huge improvement
over ext3 but for my use JFS ended up being a better fit. I
refuse/refused to go XFS.
On 12/19/2010 03:52 AM, Justin Piszcz wrote:
> Hi,
>
> I've read a lot of posts regarding people who setup RAID volumes of
> and up to around 16TB and EXT4 is typically used.
>
> However, in various forums, people still ask what is the correct
> filesystem for > 16TB? I did read one post somewhere that stated the
> ext4 developers did not recommend using ext4 for very large volumes,
> is this still true?
>
> I am looking at creating a 43TB volume possibly in the near future and
> I have used XFS in the past, which works well and would probably not
> have any problem with it; however, I have bitten quite a number of
> times by XFS bugs in the past several years, so I was curious, how
> does EXT4 perform on larger volumes, e.g., 20TB?
>
> Are there any caveats / problems?
>
> Justin.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is EXT4 the right FS for > 16TB?
2010-12-19 16:35 ` Sandon Van Ness
@ 2010-12-19 16:53 ` Justin Piszcz
2010-12-19 17:01 ` Eric Sandeen
0 siblings, 1 reply; 8+ messages in thread
From: Justin Piszcz @ 2010-12-19 16:53 UTC (permalink / raw
To: Sandon Van Ness; +Cc: linux-kernel, linux-ext4, Alan Piszcz, Eric Sandeen
Hi,
Wow, there were no updates though after Eric's last comment..
Eric, have there been any improvements in the past 6 months?
Or should one still steer clear from EXT4 > 16TB?
Justin.
On Sun, 19 Dec 2010, Sandon Van Ness wrote:
> Was it me (houkouonchi) on hard forum? I asked if > 16 TiB support was
> considered stable on here a while back:
>
> Is >16TB support considered stable?
>
> This was 6 months ago so maybe things have changed. The thread:
>
> http://kerneltrap.org/mailarchive/linux-ext4/2010/5/28/6884603/thread
>
> Luckily JFS fixed there userland utilities bug of not being able to
> handle > 32TiB very shortly after this and I ended up going that route
> and I have yet to have any data loss or problems on my JFS volume:
>
> root@dekabutsu: 08:32 AM :~# df -H /data
> Filesystem Size Used Avail Use% Mounted on
> /dev/sdd1 36T 22T 15T 61% /data
> root@dekabutsu: 08:32 AM :~#
>
> At work with our hundreds/thousands of servers we will likely be going
> ext4 as we wont be using it on >16 TiB. I think its a huge improvement
> over ext3 but for my use JFS ended up being a better fit. I
> refuse/refused to go XFS.
>
> On 12/19/2010 03:52 AM, Justin Piszcz wrote:
>> Hi,
>>
>> I've read a lot of posts regarding people who setup RAID volumes of
>> and up to around 16TB and EXT4 is typically used.
>>
>> However, in various forums, people still ask what is the correct
>> filesystem for > 16TB? I did read one post somewhere that stated the
>> ext4 developers did not recommend using ext4 for very large volumes,
>> is this still true?
>>
>> I am looking at creating a 43TB volume possibly in the near future and
>> I have used XFS in the past, which works well and would probably not
>> have any problem with it; however, I have bitten quite a number of
>> times by XFS bugs in the past several years, so I was curious, how
>> does EXT4 perform on larger volumes, e.g., 20TB?
>>
>> Are there any caveats / problems?
>>
>> Justin.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is EXT4 the right FS for > 16TB?
2010-12-19 16:53 ` Justin Piszcz
@ 2010-12-19 17:01 ` Eric Sandeen
2010-12-19 17:14 ` Ric Wheeler
2010-12-19 19:14 ` Justin Piszcz
0 siblings, 2 replies; 8+ messages in thread
From: Eric Sandeen @ 2010-12-19 17:01 UTC (permalink / raw
To: Justin Piszcz; +Cc: Sandon Van Ness, linux-kernel, linux-ext4, Alan Piszcz
On 12/19/10 10:53 AM, Justin Piszcz wrote:
> Hi,
>
> Wow, there were no updates though after Eric's last comment..
> Eric, have there been any improvements in the past 6 months?
>
> Or should one still steer clear from EXT4 > 16TB?
There is still no released e2fsprogs which supports > 16T for
ext4, but testing of the not-released bits is welcomed...
Ted says a 16T-capable version is coming soon. There's still
work to be done there, though.
-Eric
> Justin.
>
> On Sun, 19 Dec 2010, Sandon Van Ness wrote:
>
>> Was it me (houkouonchi) on hard forum? I asked if > 16 TiB support was
>> considered stable on here a while back:
>>
>> Is >16TB support considered stable?
>>
>> This was 6 months ago so maybe things have changed. The thread:
>>
>> http://kerneltrap.org/mailarchive/linux-ext4/2010/5/28/6884603/thread
>>
>> Luckily JFS fixed there userland utilities bug of not being able to
>> handle > 32TiB very shortly after this and I ended up going that route
>> and I have yet to have any data loss or problems on my JFS volume:
>>
>> root@dekabutsu: 08:32 AM :~# df -H /data
>> Filesystem Size Used Avail Use% Mounted on
>> /dev/sdd1 36T 22T 15T 61% /data
>> root@dekabutsu: 08:32 AM :~#
>>
>> At work with our hundreds/thousands of servers we will likely be going
>> ext4 as we wont be using it on >16 TiB. I think its a huge improvement
>> over ext3 but for my use JFS ended up being a better fit. I
>> refuse/refused to go XFS.
>>
>> On 12/19/2010 03:52 AM, Justin Piszcz wrote:
>>> Hi,
>>>
>>> I've read a lot of posts regarding people who setup RAID volumes of
>>> and up to around 16TB and EXT4 is typically used.
>>>
>>> However, in various forums, people still ask what is the correct
>>> filesystem for > 16TB? I did read one post somewhere that stated the
>>> ext4 developers did not recommend using ext4 for very large volumes,
>>> is this still true?
>>>
>>> I am looking at creating a 43TB volume possibly in the near future and
>>> I have used XFS in the past, which works well and would probably not
>>> have any problem with it; however, I have bitten quite a number of
>>> times by XFS bugs in the past several years, so I was curious, how
>>> does EXT4 perform on larger volumes, e.g., 20TB?
>>>
>>> Are there any caveats / problems?
>>>
>>> Justin.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is EXT4 the right FS for > 16TB?
2010-12-19 17:01 ` Eric Sandeen
@ 2010-12-19 17:14 ` Ric Wheeler
2010-12-19 19:14 ` Justin Piszcz
1 sibling, 0 replies; 8+ messages in thread
From: Ric Wheeler @ 2010-12-19 17:14 UTC (permalink / raw
To: Eric Sandeen, Justin Piszcz, Sandon Van Ness
Cc: linux-kernel, linux-ext4, Alan Piszcz
On 12/19/2010 12:01 PM, Eric Sandeen wrote:
> On 12/19/10 10:53 AM, Justin Piszcz wrote:
>> Hi,
>>
>> Wow, there were no updates though after Eric's last comment..
>> Eric, have there been any improvements in the past 6 months?
>>
>> Or should one still steer clear from EXT4> 16TB?
> There is still no released e2fsprogs which supports> 16T for
> ext4, but testing of the not-released bits is welcomed...
> Ted says a 16T-capable version is coming soon. There's still
> work to be done there, though.
>
> -Eric
I usually tend to point people towards XFS when you need something at greater
than 16TB in size....
Good luck,
Ric
>> Justin.
>>
>> On Sun, 19 Dec 2010, Sandon Van Ness wrote:
>>
>>> Was it me (houkouonchi) on hard forum? I asked if> 16 TiB support was
>>> considered stable on here a while back:
>>>
>>> Is>16TB support considered stable?
>>>
>>> This was 6 months ago so maybe things have changed. The thread:
>>>
>>> http://kerneltrap.org/mailarchive/linux-ext4/2010/5/28/6884603/thread
>>>
>>> Luckily JFS fixed there userland utilities bug of not being able to
>>> handle> 32TiB very shortly after this and I ended up going that route
>>> and I have yet to have any data loss or problems on my JFS volume:
>>>
>>> root@dekabutsu: 08:32 AM :~# df -H /data
>>> Filesystem Size Used Avail Use% Mounted on
>>> /dev/sdd1 36T 22T 15T 61% /data
>>> root@dekabutsu: 08:32 AM :~#
>>>
>>> At work with our hundreds/thousands of servers we will likely be going
>>> ext4 as we wont be using it on>16 TiB. I think its a huge improvement
>>> over ext3 but for my use JFS ended up being a better fit. I
>>> refuse/refused to go XFS.
>>>
>>> On 12/19/2010 03:52 AM, Justin Piszcz wrote:
>>>> Hi,
>>>>
>>>> I've read a lot of posts regarding people who setup RAID volumes of
>>>> and up to around 16TB and EXT4 is typically used.
>>>>
>>>> However, in various forums, people still ask what is the correct
>>>> filesystem for> 16TB? I did read one post somewhere that stated the
>>>> ext4 developers did not recommend using ext4 for very large volumes,
>>>> is this still true?
>>>>
>>>> I am looking at creating a 43TB volume possibly in the near future and
>>>> I have used XFS in the past, which works well and would probably not
>>>> have any problem with it; however, I have bitten quite a number of
>>>> times by XFS bugs in the past several years, so I was curious, how
>>>> does EXT4 perform on larger volumes, e.g., 20TB?
>>>>
>>>> Are there any caveats / problems?
>>>>
>>>> Justin.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is EXT4 the right FS for > 16TB?
2010-12-19 17:01 ` Eric Sandeen
2010-12-19 17:14 ` Ric Wheeler
@ 2010-12-19 19:14 ` Justin Piszcz
2010-12-19 19:30 ` Eric Sandeen
1 sibling, 1 reply; 8+ messages in thread
From: Justin Piszcz @ 2010-12-19 19:14 UTC (permalink / raw
To: Eric Sandeen; +Cc: Sandon Van Ness, linux-kernel, linux-ext4, Alan Piszcz
On Sun, 19 Dec 2010, Eric Sandeen wrote:
> On 12/19/10 10:53 AM, Justin Piszcz wrote:
>> Hi,
>>
>> Wow, there were no updates though after Eric's last comment..
>> Eric, have there been any improvements in the past 6 months?
>>
>> Or should one still steer clear from EXT4 > 16TB?
>
> There is still no released e2fsprogs which supports > 16T for
> ext4, but testing of the not-released bits is welcomed...
> Ted says a 16T-capable version is coming soon. There's still
> work to be done there, though.
>
> -Eric
>
Thanks Eric for confirming.
With 7 x 3TB HDD its now possible to breach 16TB (16.38TB) in RAID-5 so I
suppose more people may start asking about this.
Justin.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is EXT4 the right FS for > 16TB?
2010-12-19 19:14 ` Justin Piszcz
@ 2010-12-19 19:30 ` Eric Sandeen
2010-12-19 22:21 ` Ric Wheeler
0 siblings, 1 reply; 8+ messages in thread
From: Eric Sandeen @ 2010-12-19 19:30 UTC (permalink / raw
To: Justin Piszcz; +Cc: Sandon Van Ness, linux-kernel, linux-ext4, Alan Piszcz
On 12/19/10 1:14 PM, Justin Piszcz wrote:
>
>
> On Sun, 19 Dec 2010, Eric Sandeen wrote:
>
>> On 12/19/10 10:53 AM, Justin Piszcz wrote:
>>> Hi,
>>>
>>> Wow, there were no updates though after Eric's last comment..
>>> Eric, have there been any improvements in the past 6 months?
>>>
>>> Or should one still steer clear from EXT4 > 16TB?
>>
>> There is still no released e2fsprogs which supports > 16T for
>> ext4, but testing of the not-released bits is welcomed...
>> Ted says a 16T-capable version is coming soon. There's still
>> work to be done there, though.
>>
>> -Eric
>>
>
> Thanks Eric for confirming.
>
> With 7 x 3TB HDD its now possible to breach 16TB (16.38TB) in RAID-5 so I
> suppose more people may start asking about this.
Agreed, 16T is not that much these days.
As Ric said, XFS will handle it without problem, though.
-Eric
> Justin.
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is EXT4 the right FS for > 16TB?
2010-12-19 19:30 ` Eric Sandeen
@ 2010-12-19 22:21 ` Ric Wheeler
0 siblings, 0 replies; 8+ messages in thread
From: Ric Wheeler @ 2010-12-19 22:21 UTC (permalink / raw
To: Eric Sandeen
Cc: Justin Piszcz, Sandon Van Ness, linux-kernel, linux-ext4,
Alan Piszcz
On 12/19/2010 02:30 PM, Eric Sandeen wrote:
> On 12/19/10 1:14 PM, Justin Piszcz wrote:
>>
>> On Sun, 19 Dec 2010, Eric Sandeen wrote:
>>
>>> On 12/19/10 10:53 AM, Justin Piszcz wrote:
>>>> Hi,
>>>>
>>>> Wow, there were no updates though after Eric's last comment..
>>>> Eric, have there been any improvements in the past 6 months?
>>>>
>>>> Or should one still steer clear from EXT4> 16TB?
>>> There is still no released e2fsprogs which supports> 16T for
>>> ext4, but testing of the not-released bits is welcomed...
>>> Ted says a 16T-capable version is coming soon. There's still
>>> work to be done there, though.
>>>
>>> -Eric
>>>
>> Thanks Eric for confirming.
>>
>> With 7 x 3TB HDD its now possible to breach 16TB (16.38TB) in RAID-5 so I
>> suppose more people may start asking about this.
> Agreed, 16T is not that much these days.
>
> As Ric said, XFS will handle it without problem, though.
>
> -Eric
>
XFS as a base file system is in fact very popular with commercial storage
vendors just because of the size limitations.
Also note that will really large drives, people are also really encouraged to
use RAID-6 (larger drives take longer to rebuild, so you have more exposure to a
double failure that would lose data in RAID-5)
Ric
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2010-12-19 22:21 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-12-19 11:52 Is EXT4 the right FS for > 16TB? Justin Piszcz
2010-12-19 16:35 ` Sandon Van Ness
2010-12-19 16:53 ` Justin Piszcz
2010-12-19 17:01 ` Eric Sandeen
2010-12-19 17:14 ` Ric Wheeler
2010-12-19 19:14 ` Justin Piszcz
2010-12-19 19:30 ` Eric Sandeen
2010-12-19 22:21 ` Ric Wheeler
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.