Usando MongoDB como storage de arquivos #1

julho 20th, 2011 by caferrari Leave a reply »

Estou envolvido em um projeto que vai necessitar armazenar muitos arquivos, e precisava de uma solução que fosse tolerante a falhas e escalasse muito bem, tanto verticalmente (Mais disco) quanto horizontalmente (Mais servidores).

Depois de olhar algumas soluções existentes como MogileFS, Cassandra, entre outros, eu gostei bastante do GridFS do MongoDB, pela simplicidade de configuração e principalmente pelo sistema de eleição de um novo servidor mestre caso o ativo venha a falhar em um determinado replicaSet.

Minha configuração inicial dessa estrutura consiste em dois servidores de Storage (Ubuntu Server 11.04 64bits rodando em VMs vmware em discos diferentes) e uma terceira maquina simples e remota que será um backup onde será configurado meu primeiro ReplicaSet. No servidor de aplicação vou ter o MongoDB rodando como arbitro (sem armazenar dados) apenas para determinar qual o servidor é o mestre caso algum venha a falhar.

Instalação:

Considerando que nos servidores foi instalado o ubuntu server, sem nada, vamos a configuração:

primeiro vamos adicionar a linha na configuração do APT para podermos baixar o pacote oficial do MongoDB, mantido pela 10-gen, crie o arquivo: /etc/apt/sources.list.d/mongo.list e adicione a linha:

deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen

depois de salvar o arquivo, adicione a chave do repositorio:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

e finalmente instale o mongodb:

sudo aptitude update
sudo aptitude install mongodb-10gen

repita este processo em todas as maquinas de seu replicaSet.

Configuração do MongoDB

Primeiro vamos criar a pasta de armazenamento dos dados e definir as permissões:

sudo mkdir /dados
sudo chown mongodb:mongodb /dados/
sudo chmod 700 mongodb -Rf

Em todos os servidores vamos editar a configuração do MongoDB em /etc/mongodb.conf, aqui ficou assim:

dbpath=/dados/
logpath=/var/log/mongodb/mongodb.log
logappend=true
replSet = arquivo1
rest=true

* o parâmetro rest, faz com o que o servidor crie uma página para monitoramento dos status.

Depois de configurar todos os membros do replicaSet, reinicie o processo em todas as maquinas:

sudo service mongodb restart

Agora temos os servidores prontos para configurar a parte mais simples, o replicaset

Configuração do ReplicaSet

Para facilitar, configure nomes para seus membros do replicaSet no seu servidor de DNS interno ou, edite o arquivo /etc/hosts e adicione as maquinas que serão membro do replicaSet:

192.168.0.100      arquivo1a 
192.168.0.101      arquivo1b
192.168.0.102      arquivo1bkp

* Ajuste os IPs de acordo com sua rede.

Agora, em um servidor qualquer, acesse o console de administração do Mongo:

mongo admin

e crie a configuração do replicaSet:

conf = { _id : 'arquivo1',                    
... members : [
... {_id: 0, host: 'arquivo1a'}, 
... {_id: 1, host: 'arquivo1b'},
... {_id: 2, host: 'arquivo1bkp', priority: 0}
... ]}

* Note que o servidor de backup tem prioridade zero, isto faz com que este servidor nunca seja eleito como servidor Mestre.
confira se está tudo ok:

{
	"_id" : "arquivo1",
	"members" : [
		{
			"_id" : 0,
			"host" : "arquivo1a"
		},
		{
			"_id" : 1,
			"host" : "arquivo1b"
		},
		{
			"_id" : 2,
			"host" : "arquivo1bkp",
			"priority" : 0
		}
	]
}

se estiver ok, ainda no console do Mongo, inicialize o replicaset:

rs.initiate(conf)

em 10 segundos o seu replicaSet deverá estar rodando e pronto para receber dados, veja como está pelo navegador, acessando:

http://192.168.0.100:28017/_replSet

Você verá uma tela com o status de todos os membros do replicaSet:

Bom, é isso, no próximo post vou explicar como configurar o php e efetivamente armazenar arquivos no GridFS, Abraços

Advertisement

233 comments

  1. Une supplémentation en vitamines, minéraux et omégas 3 est une base pour
    apporter au corps l’ensemble des micronutriments conseillés.

  2. I visited ᴠarious webb sites eҳcept tɦe audio feature for audio songs existing at thiѕ webb ⲣage iss rᥱally marvelous.

    Hаve a look at myy blog post … addmefast points hack

  3. celulite disse:

    After going over a numbᥱr oof thе articles on your blog, Ⅰ really like yoսr
    technique оf blogging. I book-marked іt to my bookmark website list and wіll be checking back іn the near future.
    Plᥱase visit mу web site aѕ well and tеll me wһat you think.

    Alѕo vizit my web paցe: celulite

  4. fucking mom disse:

    Appreciating thee tіme and energy yoᥙ put іnto youur site and in depth іnformation you prеsᥱnt.
    It’s awesome to ϲome ɑcross a blog еvery оnce іn ɑ ѡhile that іsn’t tһe same old rehashed material.
    Excellent гead! I’ѵe bookmarked ʏour site and I’m adding үouг RSS feeds tߋ my Google account.

    Check oᥙt my webpage … fucking mom

  5. fireplaces 4k disse:

    Nice post. I was checking continuously this blog and I am impressed!
    Extremely helpful info specifically the last section :
    ) I care for such information much. I used to be looking for this certain information for a
    very lengthy time. Thanks and best of luck.

    My web-site fireplaces 4k

  6. La trottinette électrique légère et design de citybug avec commandes au guidon et roue gonflée.

  7. Following a rise in the vaгiеty of sgores in 2013, big bazaar telenor
    Bazar once more opened fօrty-three new stoгes in 2013,
    inclᥙdung thirty-seven within the Netherlands and six in Beⅼgium.

  8. Next, combine up a 5-gallon batch of water and Polished concrete floor finishes cleaner/degreaser in line with
    label directions (discovered at dwelling centers and hardware stores).

  9. They would have different sets of Indian bridal jewelry for each occasions custom-developed preferably by identified jewelry designers.

  10. Every weekend i used to visit this web site,
    for the reason that i want enjoyment, since this this website
    conations genuinely good funny stuff too.

  11. 13642 disse:

    Great post. I used to be checking constantly this blog and I
    am inspired! Extremely helpful info specifically the closing part :) I
    maintain such info much. I used to be looking for
    this certain information for a long time. Thanks and good luck.

  12. https://walidtourism.com

    شركات سياحية في تركيا,رحلات سياحية الى تركيا,برامج سياحية الى تركيا,رحلات داخلية في اسطنبول,تاجير سيارات مع سائق في تركيا,استئجار سيارات في اسطنبول مع سائق,رحلات شهر العسل في تركيا

  13. I’m no longer positive where you are getting your
    information, however good topic. I needs to spend some
    time finding out more or working out more. Thank you for magnificent information I was on the lookout for this info
    for my mission.

  14. http://www.almydannews.com

    الميدان , جريدة الميدان , موقع
    الميدان الاخبارى , موقع الميدان الالكترونى
    , الموقع الالكترونى لجريدة الميدان

  15. https://www.satellite-kuwait.net/

    فني ستلايت , فني هندي ستلايت , فني
    ستلايت الكويت , فني ستلايت بالكويت ,
    فني ستلايت حولي , فني ستلايت السالمية , فني ستلايت الفروانية , فني ستلايت العاصمة , فني ستلايت مبارك الكبير , فني ستلايت الجهراء , فني ستلايت الاحمدي , رقم فني ستلايت

  16. https://www.hakeemy.com

    حكيمي

    طبي ,دكتور, طب ,مستشغى ,مستشفيات, حجز موعد ,طبيب, طبيبة ,اطباء, داكترة ,معالجة ,شفاء ,حكيمي,زرع الشعر, طبيبي الخاص, طبيب
    العائلة, اسنان ,جوجل بلاي

  17. Our users send us feedback, the most popular working cheat code for EA SPORTS UFC® game was 1600 Gold :
    DyCB3ToE8oGyYil_fa”.

  18. あなたの自家用車が必ず欲しいといった中古車買取業者がいると、相場価格以上の見積もりが差しだされることも可能性としては低くはないので、
    競争式の入札で中古車一括査定サイト等をうまく活用していくことはとても重要なポイントだと思います。
    インターネットの一括車査定サービスによって、中古車の売却相場が相違します。相場価格が変わるわけは、
    そのサービス毎に、加入会社がいくらか代わるからです。
    中古車査定では、相当数のポイントがあって、それに従って弾き出された見積もり価格と、中古車買取ショップとのやり取りにより、
    買取金額が成り立っています。

  19. This is really attention-grabbing, You’re a very professional blogger.
    I’ve joined your rss feed and sit up for in quest
    of extra of your magnificent post. Additionally, I have shared your website
    in my social networks

  20. trade bearing disse:

    I have read so many articles or reviews about the blogger lovers however this article is in fact
    a nice article, keep it up.

  21. Excellent post. I was checking constantly this blog and I’m inspired!
    Very useful info particularly the remaining phase :) I handle such
    info much. I used to be looking for this certain information for a long time.
    Thanks and best of luck.

  22. Norine disse:

    That is properly above the nationwide salary average for accountants, which is around $sixty five,000.

  23. sophie evans disse:

    great ⲣoints altogether, youu simply ԝⲟn a new reader.
    What mіght yoս recommend in гegards tо your pkst tuat ʏoᥙ just made a fеw dayѕ ago?

    Any positive?

    Аlso visdit mу web site – sophie evans

  24. http://alsadanygroup.com/limousine

    ليموزين , ليموزين مطار القاهرة , ليموزين مصر, ليموزين الاسكندريه ,
    ليموزين الساحل الشمالى, ليموزين الغردقة ,
    ليموزين مطار برج العرب

  25. Ꮩery nice blog post. ӏ cerrainly apprecate tɦis website.
    Stick ԝith it!

    Herе iss my web page girls scissoring

  26. nude disse:

    Hello Dear, aree үou genuinely visiting thіѕ website daily, if ѕo
    afterward you wilkl absolսtely take nice knoѡ-how.

    Mʏ blog post nude

  27. sony earphone disse:

    Regardless of the reason why, writing thank you notes has been one habit that
    many of us has never gotten rid of through
    the years. Therefore, the music would not be as impressive to the listeners as the artists want them to be.
    After all it is your intent to say how thankful you are for the person you are giving it to, right.

  28. In Gallery 5, an installation is presented by the gallery by Kelley Johnson named Somewhere Between Here and
    There.” Here Is The performer’ sixth solo display together with
    the gallery.

  29. Ramon disse:

    What a material oof un-ambiguity and preserveness of valuable knowledgge concerning unexpected emotions.

  30. steepster.com disse:

    Unquestionably beliewve that which you said. Youur favorite reason seemed too be
    on the internet tthe easiest thing to be aware of. I say to you,
    I certainly gett annoyed while peopke consider worries that
    they plainly don’t know about. Youu managed to hit thhe nail upon the top and also defined out
    thee whole thing without having side-effects , people can take a signal.
    Will likely be back to get more. Thanks

  31. Angelica disse:

    Met specifieke zoekwoorden, LSI en SILO SEARCH ENGINE OPTIMISATION
    en de combinatie van longtail zoekwoorden kunt u de meeste en beste resultaten behalen in zoekmachines.

  32. Aucun risque de ressembler à un Malabar.

Deixe uma resposta