Usando MongoDB como storage de arquivos #1

julho 20th, 2011 by caferrari Leave a reply »

Estou envolvido em um projeto que vai necessitar armazenar muitos arquivos, e precisava de uma solução que fosse tolerante a falhas e escalasse muito bem, tanto verticalmente (Mais disco) quanto horizontalmente (Mais servidores).

Depois de olhar algumas soluções existentes como MogileFS, Cassandra, entre outros, eu gostei bastante do GridFS do MongoDB, pela simplicidade de configuração e principalmente pelo sistema de eleição de um novo servidor mestre caso o ativo venha a falhar em um determinado replicaSet.

Minha configuração inicial dessa estrutura consiste em dois servidores de Storage (Ubuntu Server 11.04 64bits rodando em VMs vmware em discos diferentes) e uma terceira maquina simples e remota que será um backup onde será configurado meu primeiro ReplicaSet. No servidor de aplicação vou ter o MongoDB rodando como arbitro (sem armazenar dados) apenas para determinar qual o servidor é o mestre caso algum venha a falhar.

Instalação:

Considerando que nos servidores foi instalado o ubuntu server, sem nada, vamos a configuração:

primeiro vamos adicionar a linha na configuração do APT para podermos baixar o pacote oficial do MongoDB, mantido pela 10-gen, crie o arquivo: /etc/apt/sources.list.d/mongo.list e adicione a linha:

deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen

depois de salvar o arquivo, adicione a chave do repositorio:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

e finalmente instale o mongodb:

sudo aptitude update
sudo aptitude install mongodb-10gen

repita este processo em todas as maquinas de seu replicaSet.

Configuração do MongoDB

Primeiro vamos criar a pasta de armazenamento dos dados e definir as permissões:

sudo mkdir /dados
sudo chown mongodb:mongodb /dados/
sudo chmod 700 mongodb -Rf

Em todos os servidores vamos editar a configuração do MongoDB em /etc/mongodb.conf, aqui ficou assim:

dbpath=/dados/
logpath=/var/log/mongodb/mongodb.log
logappend=true
replSet = arquivo1
rest=true

* o parâmetro rest, faz com o que o servidor crie uma página para monitoramento dos status.

Depois de configurar todos os membros do replicaSet, reinicie o processo em todas as maquinas:

sudo service mongodb restart

Agora temos os servidores prontos para configurar a parte mais simples, o replicaset

Configuração do ReplicaSet

Para facilitar, configure nomes para seus membros do replicaSet no seu servidor de DNS interno ou, edite o arquivo /etc/hosts e adicione as maquinas que serão membro do replicaSet:

192.168.0.100      arquivo1a 
192.168.0.101      arquivo1b
192.168.0.102      arquivo1bkp

* Ajuste os IPs de acordo com sua rede.

Agora, em um servidor qualquer, acesse o console de administração do Mongo:

mongo admin

e crie a configuração do replicaSet:

conf = { _id : 'arquivo1',                    
... members : [
... {_id: 0, host: 'arquivo1a'}, 
... {_id: 1, host: 'arquivo1b'},
... {_id: 2, host: 'arquivo1bkp', priority: 0}
... ]}

* Note que o servidor de backup tem prioridade zero, isto faz com que este servidor nunca seja eleito como servidor Mestre.
confira se está tudo ok:

{
	"_id" : "arquivo1",
	"members" : [
		{
			"_id" : 0,
			"host" : "arquivo1a"
		},
		{
			"_id" : 1,
			"host" : "arquivo1b"
		},
		{
			"_id" : 2,
			"host" : "arquivo1bkp",
			"priority" : 0
		}
	]
}

se estiver ok, ainda no console do Mongo, inicialize o replicaset:

rs.initiate(conf)

em 10 segundos o seu replicaSet deverá estar rodando e pronto para receber dados, veja como está pelo navegador, acessando:

http://192.168.0.100:28017/_replSet

Você verá uma tela com o status de todos os membros do replicaSet:

Bom, é isso, no próximo post vou explicar como configurar o php e efetivamente armazenar arquivos no GridFS, Abraços

Advertisement

270 comments

  1. Hi there, after reading this remarkable paragraph i am
    as well cheerful to share my experience here with friends.

  2. If some one desires to be updated with most recent technologies therefore he must be
    go to see this website and be up to date daily.

  3. A loja vende tanto no atacado quanto no varejo e
    tudo direto da China, roupas a preços muito competitivos,
    e entrega para mundo todo.

  4. tire isso de mim e também perdoe-me se peco, La historia nos demuestra que las reinas
    y princesas, a secção de representar a su país, son consideradas verdaderos iconos de estilo.
    está incorrer no 7º vício mortal, se não for em outros, pois, ociosidade poderá
    ser uma porta ensejo para outros pecados, já do corpo ou do alma e também, conforme

  5. Chad disse:

    2. One other necessary issue is reliability.

    my blog – 3883connave.com; Chad,

  6. We are unable to uncover iTunes on your laptop. To get and download
    Complete Motion pictures – powered by YouTube by Stefano Mendicino, get iTunes now.

  7. coc mod disse:

    Feel free tto visit my weblog: coc mod

  8. Este soube e viu nosso fruto pelas fotos porém nunca nem se importou meu fruto vem tentando, fazer
    faculdade ja trancou varias vezes por falta de condições será que a justiça poderia obriga-lo a ajudar
    já que não fez a vida toda, meu filho hoje tem 24 anos, e possui uma filha.
    tire isso de mim e também perdoe-me se peco, está incorrer no 7º
    pecado mortal, se não for em outros, porque, repouso poderá
    ser uma porta oportunidade para outros pecados, ou do corpo ora do espírito e, conforme

  9. Without wasting your own time if the website doesn’t have it, you’re able to basically move
    to another.

  10. Hello, i think that i noticed you visited my site thus i got here to return the choose?.I’m attempting to find things to enhance
    my site!I guess its good enough to make use of a few of your ideas!!

  11. tire isso de mim e também perdoe-me se peco,
    Quanto aos produtos, Brylcreem se você estiver indo fé. Ouvi expressar
    que eles usam Baxter da Califórnia Pomada ou
    Gel no set de Mad Men. está incorrer no 7º perversão
    mortal, se não for em outros, pois, repouso poderá ser uma porta aberta para outros pecados, ora do corpo ou do anjo e também, conforme E eu apreciaria de que você dissesse pra
    gente em relação a atração, como você disse, não só da sedução ou da
    pegação ou nenhum gênero de coisa de forma, porém da atração de coisas boas, controlar
    da imagem.

  12. Hello great website! Does running a blog such as this require a lot of work?
    I have no knowledge of computer programming however I had been hoping to start
    my own blog soon. Anyways, should you have any ideas or tips for new blog owners please share.
    I know this is off subject nevertheless I simply had to ask.
    Thanks a lot!

  13. For latest information you have to go to see world-wide-web and on internet I
    found this web page as a most excellent website for
    hottest updates.

  14. Essa organizadora está sendo responsáveis pela maior número dos concursos, na área com ensino,
    requisitada pela Prefeitura do Recife e também pelo governo do Estado a Pernambuco.

  15. I just like the valuable information you supply to your
    articles. I’ll bookmark your weblog and test again here frequently.

    I’m quite certain I will learn many new stuff right here!

    Good luck for the following!

  16. Quality content is the main to invite the users to visit the website,
    that’s what this web site is providing.

  17. Mercedes disse:

    I was wondering if you ever thought of changing the layout of your blog?

    Its very well written; I love what youve got to say. But maybe you could a little more
    in the way of content so people could connect with it better.
    Youve got an awful lot of text for only having one or 2 pictures.
    Maybe you could space it out better?

  18. A. Do not even consider calling her around the clock
    - seriously, this will certainly get on her nerves and she will
    of course be checked off.

  19. talk to women disse:

    When your girlfriend or other half is unhappy in your relationship, you
    ‘d much better beware because females tend to have affairs when they feel
    detached from the marital relationship.

  20. Optimisez votre recherche en prenant le temps de consulter les
    avis conso des fidèles customers du leader du e-commerce en France
    avant de passer votre commande de cuisinière gaz pas cher.

  21. 茨城県のダイヤモンド買取の後押しをレポート。善くサイトです。

  22. Zu erwähnen іst natürlich diᥱ Autochartistik, ԁіe vom Überblick sehг gute Leistungen als Unterstützung für ԁіе Betrachtung
    anbieten kann.

  23. Games: I appeared on the Minute to Win It web site and tried to choose video games I assumed seemed enjoyable, and that I had the provides for.

  24. If you would like to obtain a great deal from this article then you have to apply such techniques to your won blog.

  25. When searching for a good dentist, people need to search for someone who
    loves his job and is exceptional at it; the search procedure ought to not
    be too hard, for the majority of dental practitioners market
    their services on the Internet and post a CV where people can discover comprehensive info
    about their studies and experience in the field of dentistry.
    There have actually been plenty of innovative technological developments in the field of dentistry that in fact make going to the dentist
    pleasurable.

  26. لايفوتك,موقع لايفوتك , تطبيق لايفوتك

    http://www.layfotak.com

  27. http://al-anoud.net/

    كشف تسربات المياه بالدمام ,
    عزل اسطح بالدمام ,شركة كشف تسربات المياه
    بالدمام , افضل شركة كشف تسربات المياه
    بالدمام , شركة عزل اسطح بالدمام

  28. Hurrah, that’s what I was searching for,
    what a data! present here at this weblog, thanks admin of this website.

  29. Awesome site you have here but I was curious about if you knew
    of any message boards that cover the same topics discussed here?
    I’d really like to be a part of community where I can get opinions
    from other experienced people that share the same interest.
    If you have any suggestions, please let me know. Many thanks!

  30. ᕼello my family member! I wish to say
    that thіs post is awesome, great written and include approximatᥱly all significant infos.
    I’d like to look extгa poѕts like this .

    mу web page купить кисти sigma

  31. montreal dsl disse:

    The Worldwide Cable Protection Committee, an association of submwrine cable operators, said on its web site it
    was aware of multiple submarine cable failures affectig Web
    speeds on some routes however did not knoww what had brought on tthe issue.

    Feel free tto visit my site … montreal dsl

  32. Hmm it seems like your blog ate my first comment (it was super long) so I
    guess I’ll just sum it up what I submitted and say, I’m thoroughly enjoying your blog.
    I as well am an aspiring blog writer but I’m still new to everything.
    Do you have any tips for novice blog writers? I’d certainly appreciate it.

  33. http://www.almesria.com

    شقق للايجار بالتجمع الخامس,شقة للايجار بالتجمع الخامس,
    شقق للايجار فى التجمع الخامس,شقة للايجار فى التجمع الخامس,فيلا للايجار بالتجمع الخامس,فيلا للايجار فى
    التجمع الخامس,شقة للبيع بمشروع دار مصر,شقة للبيع بدار مصر التجمع الخامس,شقق للبيع بالتجمع الخامس,شقة للبيع بالتجمع الخامس,,شقة للبيع بكمبوند فى التجمع الخامس,شقة
    للبيع داخل كمبوند فى التجمع الخامس ,ارض للبيع بالتجمع الخامس,ارض للبيع فى التجمع الخامس,اراضى للبيع بالتجمع الخامس,اراضى للبيع فى التجمع الخامس

  34. Thanks for sharing your thoughts about sudski tumac
    za turski jezik. Regards

  35. Lenovo VIBE P1, gücünü Snapdragon 615 işlemcisinden alırken cihazda ayrıca Adreno 405 GPU, 3 GB RAM ve 32 GB dahili depolama alanı yer alıyor.

  36. What’s up, its fastidious article about media print, we all be
    familiar with media is a wonderful source of facts.

  37. 神奈川県の着物処分を評注するよ。持ちこたえる言及致すします。

  38. Perhaps it’s better that they focus on making nice motion pictures as an alternative of middling video video games.

  39. 富山県の着物査定の引き当てるケチるとは。いやいや御遠慮なくいいな。

  40. jadeite jade disse:

    whoah this weblog is excellent i love reading your articles.
    Stay up the good work! You know, many persons are looking round for
    this info, you could help them greatly.

  41. http://kiddie-pediatrics.com/

    دكتور اطفال فى مصر الجديدة,احسن دكتور اطفال فى مصر الجديدة,افضل دكتور اطفال فى مصر الجديدة,افضل دكتور اطفال فى مصر ,اشطر دكتور اطفال,دكتور حساسية ومناعة اطفال ,
    دكتور كلي اطفال ,سمنة الاطفال,حساسية
    الالبان,حساسية الصدر,المتلازمة النفروزية,العيوب الخلقية,تشوهات الكلى والمسالك
    ,علاج سمنة الاطفال,علاج حساسية الالبان عند الاطفال,علاج حساسية
    الصدر عند الاطفال,علاج المتلازمة النفروزية
    عند الاطفال,علاج العيوب الخلقية عند الاطفال ,علاج تشوهات
    الكلى والمسالك البولية عند الاطفال ,علاج امراض دم الاطفال، حساسية و مناعة اطفال,علاج روماتيزم
    الاطفال,علاج الحالات الحرجة للاطفال ,متابعة شهرية لنمو الاطفال,
    علاج النزلات المعويه ,علاج حالات التبول اللاارادي عند الاطفال,التطعيمات الكاملة للاطفال ,تشخيص و علاج حالات السمنة و النحافة في الأطفال,تشخيص الاضطرابات السلوكية و النفسية للأطفال,تشخيص و علاج ارتفاع الصفراء في حديثي الولادة ,علاج و متابعة نقص النمو ,متابعة شهرية لنمو الاطفال,

  42. This page certainly has all the information I needed about this subject and didn’t know who
    to ask.

  43. In contrast fitness trackers with heart rate and gps copper, the value of 1
    kkm optical fiber is about eighty five yuan -90 yuan, the value
    of copper has been decreas than the price of mileage.

  44. Kieran disse:

    As quickly as your interlocking paving stones
    are put in, you can instantly stroll or drive over them.

  45. What’s up colleagues, һow is the ᴡhole tһing, and ԝhat ʏou esire tоo say
    on the topic ⲟf thiѕ article, іn mү vieѡ
    itѕ іn faсt awesome іn support ⲟf me.

    Here iis my page; making money from home

  46. I know this web page offers quality depending articles or reviews
    and extra information, is there any other web site which presents these
    kinds of things in quality?

  47. Just desire tо saү үour article is as amazing. The clearness on yoour ρut
    up is just ցreat and thɑt i could suppose yoս’rе a professional on thіs subject.
    Fіne along witһ your permission ⅼet me to grab yoսr RSS feed
    to stay ᥙp too daate with approaching post. Thamk үou one millіon and please keеp up the gratifying աork.

    Aⅼso visit mmy pagе: Free Website Hits

  48. example disse:

    Hi, after reading this amazing article i am also cheerful
    to share my know-how here with friends.

  49. Hi there friends, how is all, and what you want to say on the topic of this post,
    in my view its genuinely remarkable in support of me.

Trackbacks /
Pingbacks

Deixe uma resposta